[jira] [Work logged] (HIVE-21109) Stats replication for ACID tables.
[
https://issues.apache.org/jira/browse/HIVE-21109?focusedWorklogId=225969=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-225969
]
ASF GitHub Bot logged work on HIVE-21109:
-
Author: ASF GitHub Bot
Created on: 11/Apr/19 05:41
Start Date: 11/Apr/19 05:41
Worklog Time Spent: 10m
Work Description: ashutosh-bapat commented on pull request #579:
HIVE-21109 : Support stats replication for ACID tables.
URL: https://github.com/apache/hive/pull/579#discussion_r274264405
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMM.java
##
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.parse;
+
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.metastore.conf.MetastoreConf;
+import
org.apache.hadoop.hive.metastore.messaging.json.gzip.GzipJSONMessageEncoder;
+import org.junit.BeforeClass;
+import org.junit.Rule;
+import org.junit.rules.TestName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Tests statistics replication for ACID tables.
+ */
+public class TestStatsReplicationScenariosMM extends
TestStatsReplicationScenarios {
+ @Rule
+ public final TestName testName = new TestName();
+
+ @BeforeClass
+ public static void classLevelSetup() throws Exception {
+Map overrides = new HashMap<>();
+overrides.put(MetastoreConf.ConfVars.EVENT_MESSAGE_FACTORY.getHiveName(),
+GzipJSONMessageEncoder.class.getCanonicalName());
+overrides.put(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY.varname, "true");
+overrides.put(HiveConf.ConfVars.HIVE_TXN_MANAGER.varname,
+ "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager");
+
overrides.put(MetastoreConf.ConfVars.CAPABILITY_CHECK.getHiveName(),"false");
+
overrides.put(HiveConf.ConfVars.REPL_BOOTSTRAP_DUMP_OPEN_TXN_TIMEOUT.varname,"1s");
+overrides.put(HiveConf.ConfVars.DYNAMICPARTITIONINGMODE.varname,
"nonstrict");
+
+
+internalBeforeClassSetup(overrides, overrides,
TestReplicationScenarios.class, true, "mm");
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 225969)
Time Spent: 13h 20m (was: 13h 10m)
> Stats replication for ACID tables.
> --
>
> Key: HIVE-21109
> URL: https://issues.apache.org/jira/browse/HIVE-21109
> Project: Hive
> Issue Type: Sub-task
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21109.01.patch, HIVE-21109.02.patch,
> HIVE-21109.03.patch, HIVE-21109.04.patch, HIVE-21109.05.patch,
> HIVE-21109.06.patch, HIVE-21109.07.patch, HIVE-21109.08.patch,
> HIVE-21109.09.patch, HIVE-21109.09.patch, HIVE-21109.10.patch
>
> Time Spent: 13h 20m
> Remaining Estimate: 0h
>
> Transactional tables require a writeID associated with the stats update. This
> writeId needs to be in sync with the writeId on the source and hence needs to
> be replicated from the source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21109) Stats replication for ACID tables.
[
https://issues.apache.org/jira/browse/HIVE-21109?focusedWorklogId=225970=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-225970
]
ASF GitHub Bot logged work on HIVE-21109:
-
Author: ASF GitHub Bot
Created on: 11/Apr/19 05:41
Start Date: 11/Apr/19 05:41
Worklog Time Spent: 10m
Work Description: ashutosh-bapat commented on pull request #579:
HIVE-21109 : Support stats replication for ACID tables.
URL: https://github.com/apache/hive/pull/579#discussion_r274264424
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMMNoAutogather.java
##
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.parse;
+
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.metastore.conf.MetastoreConf;
+import
org.apache.hadoop.hive.metastore.messaging.json.gzip.GzipJSONMessageEncoder;
+import org.junit.BeforeClass;
+import org.junit.Rule;
+import org.junit.rules.TestName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Tests statistics replication for ACID tables.
+ */
+public class TestStatsReplicationScenariosMMNoAutogather extends
TestStatsReplicationScenarios {
+ @Rule
+ public final TestName testName = new TestName();
+
+ @BeforeClass
+ public static void classLevelSetup() throws Exception {
+Map overrides = new HashMap<>();
+overrides.put(MetastoreConf.ConfVars.EVENT_MESSAGE_FACTORY.getHiveName(),
+GzipJSONMessageEncoder.class.getCanonicalName());
+overrides.put(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY.varname, "true");
+overrides.put(HiveConf.ConfVars.HIVE_TXN_MANAGER.varname,
+ "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager");
+
overrides.put(MetastoreConf.ConfVars.CAPABILITY_CHECK.getHiveName(),"false");
+
overrides.put(HiveConf.ConfVars.REPL_BOOTSTRAP_DUMP_OPEN_TXN_TIMEOUT.varname,"1s");
+overrides.put(HiveConf.ConfVars.DYNAMICPARTITIONINGMODE.varname,
"nonstrict");
+overrides.put("mapred.input.dir.recursive", "true");
+
+
+internalBeforeClassSetup(overrides, overrides,
TestReplicationScenarios.class, false, "mm");
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 225970)
Time Spent: 13.5h (was: 13h 20m)
> Stats replication for ACID tables.
> --
>
> Key: HIVE-21109
> URL: https://issues.apache.org/jira/browse/HIVE-21109
> Project: Hive
> Issue Type: Sub-task
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21109.01.patch, HIVE-21109.02.patch,
> HIVE-21109.03.patch, HIVE-21109.04.patch, HIVE-21109.05.patch,
> HIVE-21109.06.patch, HIVE-21109.07.patch, HIVE-21109.08.patch,
> HIVE-21109.09.patch, HIVE-21109.09.patch, HIVE-21109.10.patch
>
> Time Spent: 13.5h
> Remaining Estimate: 0h
>
> Transactional tables require a writeID associated with the stats update. This
> writeId needs to be in sync with the writeId on the source and hence needs to
> be replicated from the source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21109) Stats replication for ACID tables.
[
https://issues.apache.org/jira/browse/HIVE-21109?focusedWorklogId=225966=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-225966
]
ASF GitHub Bot logged work on HIVE-21109:
-
Author: ASF GitHub Bot
Created on: 11/Apr/19 05:36
Start Date: 11/Apr/19 05:36
Worklog Time Spent: 10m
Work Description: ashutosh-bapat commented on pull request #579:
HIVE-21109 : Support stats replication for ACID tables.
URL: https://github.com/apache/hive/pull/579#discussion_r274263457
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenarios.java
##
@@ -224,6 +236,17 @@ private void
verifyNoPartitionStatsReplicationForMetadataOnly(String tableName)
}
}
+ private String getCreateTableProperties() {
+if (acidTableKindToUse != null) {
+ if (acidTableKindToUse.equals("orc")) {
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 225966)
Time Spent: 13h 10m (was: 13h)
> Stats replication for ACID tables.
> --
>
> Key: HIVE-21109
> URL: https://issues.apache.org/jira/browse/HIVE-21109
> Project: Hive
> Issue Type: Sub-task
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21109.01.patch, HIVE-21109.02.patch,
> HIVE-21109.03.patch, HIVE-21109.04.patch, HIVE-21109.05.patch,
> HIVE-21109.06.patch, HIVE-21109.07.patch, HIVE-21109.08.patch,
> HIVE-21109.09.patch, HIVE-21109.09.patch, HIVE-21109.10.patch
>
> Time Spent: 13h 10m
> Remaining Estimate: 0h
>
> Transactional tables require a writeID associated with the stats update. This
> writeId needs to be in sync with the writeId on the source and hence needs to
> be replicated from the source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21602) Dropping an external table created by migration case should delete the data directory.
[ https://issues.apache.org/jira/browse/HIVE-21602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan reassigned HIVE-21602: --- > Dropping an external table created by migration case should delete the data > directory. > -- > > Key: HIVE-21602 > URL: https://issues.apache.org/jira/browse/HIVE-21602 > Project: Hive > Issue Type: Bug > Components: repl >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication > > For external table, if the table is dropped, the location is not removed. But > If the source table is managed and at target the table is converted to > external, then the table location should be removed if the table is dropped. > Replication flow should set additional parameter > "external.table.purge"="true" for migration to external table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21500) Disable conversion of managed table to external and vice versa at source.
[ https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21500: Status: Patch Available (was: Open) Fixed checkstyle issues. > Disable conversion of managed table to external and vice versa at source. > - > > Key: HIVE-21500 > URL: https://issues.apache.org/jira/browse/HIVE-21500 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication, pull-request-available > Attachments: HIVE-21500.01.patch, HIVE-21500.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) > replication where managed table is converted to external at source. > *Scenario-1: (ACID/MM table converted to external at target)* > 1. Create non-ACID ORC format table. > 2. Insert some rows > 3. Replicate this create event which creates ACID table at target (due to > migration rule). Each insert event adds transactional metadata in HMS > corresponding to the current table. > 4. Convert table to external table using ALTER command at source. > *Scenario-2: (External table at target changes table location)* > 1. Create non-ACID avro format table. > 2. Insert some rows > 3. Replicate this create event which creates external table at target (due to > migration rule). The data path is chosen under default external warehouse > directory. > 4. Convert table to external table using ALTER command at source. > It is unable to convert an ACID table to external table at target. Also, it > is hard to detect what would be the table type at target when perform this > ALTER table operation at source. > So, it is decided to disable conversion of managed table at source (Hive2) to > EXTERNAL or vice-versa if the DB is enabled for replication and strict > managed is disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21500) Disable conversion of managed table to external and vice versa at source.
[ https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21500: Attachment: HIVE-21500.02.patch > Disable conversion of managed table to external and vice versa at source. > - > > Key: HIVE-21500 > URL: https://issues.apache.org/jira/browse/HIVE-21500 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication, pull-request-available > Attachments: HIVE-21500.01.patch, HIVE-21500.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) > replication where managed table is converted to external at source. > *Scenario-1: (ACID/MM table converted to external at target)* > 1. Create non-ACID ORC format table. > 2. Insert some rows > 3. Replicate this create event which creates ACID table at target (due to > migration rule). Each insert event adds transactional metadata in HMS > corresponding to the current table. > 4. Convert table to external table using ALTER command at source. > *Scenario-2: (External table at target changes table location)* > 1. Create non-ACID avro format table. > 2. Insert some rows > 3. Replicate this create event which creates external table at target (due to > migration rule). The data path is chosen under default external warehouse > directory. > 4. Convert table to external table using ALTER command at source. > It is unable to convert an ACID table to external table at target. Also, it > is hard to detect what would be the table type at target when perform this > ALTER table operation at source. > So, it is decided to disable conversion of managed table at source (Hive2) to > EXTERNAL or vice-versa if the DB is enabled for replication and strict > managed is disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21500) Disable conversion of managed table to external and vice versa at source.
[ https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21500: Status: Open (was: Patch Available) > Disable conversion of managed table to external and vice versa at source. > - > > Key: HIVE-21500 > URL: https://issues.apache.org/jira/browse/HIVE-21500 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication, pull-request-available > Attachments: HIVE-21500.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) > replication where managed table is converted to external at source. > *Scenario-1: (ACID/MM table converted to external at target)* > 1. Create non-ACID ORC format table. > 2. Insert some rows > 3. Replicate this create event which creates ACID table at target (due to > migration rule). Each insert event adds transactional metadata in HMS > corresponding to the current table. > 4. Convert table to external table using ALTER command at source. > *Scenario-2: (External table at target changes table location)* > 1. Create non-ACID avro format table. > 2. Insert some rows > 3. Replicate this create event which creates external table at target (due to > migration rule). The data path is chosen under default external warehouse > directory. > 4. Convert table to external table using ALTER command at source. > It is unable to convert an ACID table to external table at target. Also, it > is hard to detect what would be the table type at target when perform this > ALTER table operation at source. > So, it is decided to disable conversion of managed table at source (Hive2) to > EXTERNAL or vice-versa if the DB is enabled for replication and strict > managed is disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21538) Beeline: password source though the console reader did not pass to connection param
[
https://issues.apache.org/jira/browse/HIVE-21538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815076#comment-16815076
]
Rajkumar Singh commented on HIVE-21538:
---
[~thejas] I have done the suggested change, could you please review. Thanks
> Beeline: password source though the console reader did not pass to connection
> param
> ---
>
> Key: HIVE-21538
> URL: https://issues.apache.org/jira/browse/HIVE-21538
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
> Environment: Hive-3.1 auth set to LDAP
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-21538.01.patch, HIVE-21538.02.patch,
> HIVE-21538.patch
>
>
> Beeline: password source through the console reader do not pass to connection
> param, this will yield into the Authentication failure in case of LDAP
> authentication.
> {code}
> beeline -n USER -u
> "jdbc:hive2://host:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2"
> -p
> Connecting to
> jdbc:hive2://host:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;user=USER
> Enter password for jdbc:hive2://host:2181/:
> 19/03/26 19:49:44 [main]: WARN jdbc.HiveConnection: Failed to connect to
> host:1
> 19/03/26 19:49:44 [main]: ERROR jdbc.Utils: Unable to read HiveServer2
> configs from ZooKeeper
> Unknown HS2 problem when communicating with Thrift server.
> Error: Could not open client transport for any of the Server URI's in
> ZooKeeper: Peer indicated failure: PLAIN auth failed:
> javax.security.sasl.AuthenticationException: Error validating LDAP user
> [Caused by javax.naming.AuthenticationException: [LDAP: error code 49 -
> 80090308: LdapErr: DSID-0C0903C8, comment: AcceptSecurityContext error, data
> 52e, v2580]] (state=08S01,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21601) Hive JDBC Storage Handler query fail because projected timestamp max precision is not valid for mysql
[
https://issues.apache.org/jira/browse/HIVE-21601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh updated HIVE-21601:
--
Description:
Steps to reproduce:
{code}
--mysql table
mysql> show create table dd_timestamp_error;
++--+
| Table | Create Table
|
++--+
| dd_timestamp_error | CREATE TABLE `dd_timestamp_error` (
`col1` text,
`col2` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE
CURRENT_TIMESTAMP(6)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
++--+
1 row in set (0.00 sec)
-- hive table
++
| createtab_stmt |
++
| CREATE EXTERNAL TABLE `dd_timestamp_error`(|
| `col1` string COMMENT 'from deserializer', |
| `col2` timestamp COMMENT 'from deserializer')|
| ROW FORMAT SERDE |
| 'org.apache.hive.storage.jdbc.JdbcSerDe' |
| STORED BY |
| 'org.apache.hive.storage.jdbc.JdbcStorageHandler' |
| WITH SERDEPROPERTIES ( |
| 'serialization.format'='1') |
| TBLPROPERTIES (|
| 'bucketing_version'='2', |
| 'hive.sql.database.type'='MYSQL',|
| 'hive.sql.dbcp.maxActive'='1', |
| 'hive.sql.dbcp.password'='testuser', |
| 'hive.sql.dbcp.username'='testuser', |
| 'hive.sql.jdbc.driver'='com.mysql.jdbc.Driver', |
| 'hive.sql.jdbc.url'='jdbc:mysql://c46-node3.squadron-labs.com/test', |
| 'hive.sql.table'='dd_timestamp_error', |
| 'transient_lastDdlTime'='1554910389')|
++
--query failure
0: jdbc:hive2://c46-node2.squadron-labs.com:2> select * from
dd_timestamp_error where col2 = '2019-04-03 15:54:21.543654';
Error: java.io.IOException: java.io.IOException:
org.apache.hive.storage.jdbc.exception.HiveJdbcDatabaseAccessException: Caught
exception while trying to execute query:You have an error in your SQL syntax;
check the manual that corresponds to your MySQL server version for the right
syntax to use near 'TIMESTAMP(9)) AS `col2`
--
explain select * from dd_timestamp_error where col2 = '2019-04-03
15:54:21.543654';
TableScan [TS_0] |
| Output:["col1","col2"],properties:{"hive.sql.query":"SELECT `col1`,
CAST(TIMESTAMP '2019-04-03 15:54:21.543654000' AS TIMESTAMP(9)) AS `col2`\nFROM
`dd_timestamp_error`\nWHERE `col2` = TIMESTAMP '2019-04-03
15:54:21.543654000'","hive.sql.query.fieldNames":"col1,col2","hive.sql.query.fieldTypes":"string,timestamp","hive.sql.query.split":"true"}
|
|
{code}
the problem seems to be with convertedFilterExpr ( -- where col2 = '2019-04-03
15:54:21.543654';) while comparing timestamp with constant:-
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java#L856
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveTypeSystemImpl.java#L38
hive timestamp MAX_TIMESTAMP_PRECISION seems to be 9 and it appears that hive
pushes the same in query projection(JDBC project) for MySQL and fail the query
since max timestamp precision in MySQL is 6.
was:
Steps to reproduce:
{code}
--mysql table
mysql> show create table dd_timestamp_error;
++--+
| Table | Create Table
|
++--+
[jira] [Updated] (HIVE-21164) ACID: explore how we can avoid a move step during inserts/compaction
[ https://issues.apache.org/jira/browse/HIVE-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-21164: Attachment: HIVE-21164.1.patch > ACID: explore how we can avoid a move step during inserts/compaction > > > Key: HIVE-21164 > URL: https://issues.apache.org/jira/browse/HIVE-21164 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-21164.1.patch > > > Currently, we write compacted data to a temporary location and then move the > files to a final location, which is an expensive operation on some cloud file > systems. Since HIVE-20823 is already in, it can control the visibility of > compacted data for the readers. Therefore, we can perhaps avoid writing data > to a temporary location and directly write compacted data to the intended > final path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21600) GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS with multiple children
[
https://issues.apache.org/jira/browse/HIVE-21600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815021#comment-16815021
]
Hive QA commented on HIVE-21600:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965531/HIVE-21600.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15896 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16920/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16920/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16920/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965531 - PreCommit-HIVE-Build
> GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS
> with multiple children
> --
>
> Key: HIVE-21600
> URL: https://issues.apache.org/jira/browse/HIVE-21600
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21600.patch, HIVE-21600.patch
>
>
> The method does not reset the context when it loops through the children of
> TS. Hence, if TS has multiple FIL children, we can end up with a mangled
> context.
> Stacktrace:
> {code}
> ERROR : FAILED: ArrayIndexOutOfBoundsException -1
> java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:418)
> at java.util.ArrayList.remove(ArrayList.java:495)
> at
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeSemiJoinOperator(GenTezUtils.java:653)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.sharedWorkOptimization(SharedWorkOptimizer.java:452)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.transform(SharedWorkOptimizer.java:151)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:186)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12495)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:362)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:172)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1870)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1817)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1812)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21600) GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS with multiple children
[
https://issues.apache.org/jira/browse/HIVE-21600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815004#comment-16815004
]
Hive QA commented on HIVE-21600:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
12s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16920/dev-support/hive-personality.sh
|
| git revision | master / 6e7e299 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16920/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS
> with multiple children
> --
>
> Key: HIVE-21600
> URL: https://issues.apache.org/jira/browse/HIVE-21600
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21600.patch, HIVE-21600.patch
>
>
> The method does not reset the context when it loops through the children of
> TS. Hence, if TS has multiple FIL children, we can end up with a mangled
> context.
> Stacktrace:
> {code}
> ERROR : FAILED: ArrayIndexOutOfBoundsException -1
> java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:418)
> at java.util.ArrayList.remove(ArrayList.java:495)
> at
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeSemiJoinOperator(GenTezUtils.java:653)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.sharedWorkOptimization(SharedWorkOptimizer.java:452)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.transform(SharedWorkOptimizer.java:151)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:186)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12495)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:362)
> at
>
[jira] [Commented] (HIVE-21584) Java 11 preparation: system class loader is not URLClassLoader
[
https://issues.apache.org/jira/browse/HIVE-21584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814979#comment-16814979
]
Hive QA commented on HIVE-21584:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965525/HIVE-21584.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15902 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16919/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16919/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16919/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965525 - PreCommit-HIVE-Build
> Java 11 preparation: system class loader is not URLClassLoader
> --
>
> Key: HIVE-21584
> URL: https://issues.apache.org/jira/browse/HIVE-21584
> Project: Hive
> Issue Type: Task
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Major
> Attachments: HIVE-21584.01.patch, HIVE-21584.02.patch,
> HIVE-21584.03.patch
>
>
> Currently, Hive assumes that the system class loader is instance of
> {{URLClassLoader}}. In Java 11 this is not the case. There are a few
> (unresolved) JIRAs about specific occurrences of {{URLClassLoader}} (e.g.
> [HIVE-21237|https://issues.apache.org/jira/browse/HIVE-21237],
> [HIVE-17909|https://issues.apache.org/jira/browse/HIVE-17909]), but no
> _"remove all occurrences"_. Also I couldn't find umbrella "Java 11 upgrade"
> JIRA.
> This ticket is to remove all unconditional casts of any random class loader
> to {{URLClassLoader}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21584) Java 11 preparation: system class loader is not URLClassLoader
[
https://issues.apache.org/jira/browse/HIVE-21584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814970#comment-16814970
]
Hive QA commented on HIVE-21584:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
51s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
30s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} spark-client in master has 10 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} llap-server in master has 81 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} beeline in master has 44 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} standalone-metastore/metastore-common: The patch
generated 0 new + 22 unchanged - 2 fixed = 22 total (was 24) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch spark-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} ql: The patch generated 0 new + 196 unchanged - 1
fixed = 196 total (was 197) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} llap-server: The patch generated 0 new + 8 unchanged
- 1 fixed = 8 total (was 9) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch beeline passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
36s{color} | {color:green} metastore-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} spark-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} ql generated 0 new + 2254 unchanged - 4 fixed = 2254
total (was 2258) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} llap-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} |
[jira] [Commented] (HIVE-21533) Nested CTE's with join does not return any data.
[
https://issues.apache.org/jira/browse/HIVE-21533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814952#comment-16814952
]
Vineet Garg commented on HIVE-21533:
[~ngangam] Is this only reproducible with 2.1? On upstream I am getting data.
{noformat}
201409 201409 AAA 2 201901 201409 1
{noformat}
> Nested CTE's with join does not return any data.
>
>
> Key: HIVE-21533
> URL: https://issues.apache.org/jira/browse/HIVE-21533
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 2.1.0
>Reporter: Naveen Gangam
>Priority: Major
> Attachments: testcase.sql
>
>
> Attached is the testcase to reproduce the issue. the join on CTE6 is causing
> the problem.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21600) GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS with multiple children
[
https://issues.apache.org/jira/browse/HIVE-21600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-21600:
---
Attachment: HIVE-21600.patch
> GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS
> with multiple children
> --
>
> Key: HIVE-21600
> URL: https://issues.apache.org/jira/browse/HIVE-21600
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21600.patch, HIVE-21600.patch
>
>
> The method does not reset the context when it loops through the children of
> TS. Hence, if TS has multiple FIL children, we can end up with a mangled
> context.
> Stacktrace:
> {code}
> ERROR : FAILED: ArrayIndexOutOfBoundsException -1
> java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:418)
> at java.util.ArrayList.remove(ArrayList.java:495)
> at
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeSemiJoinOperator(GenTezUtils.java:653)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.sharedWorkOptimization(SharedWorkOptimizer.java:452)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.transform(SharedWorkOptimizer.java:151)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:186)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12495)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:362)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:172)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1870)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1817)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1812)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21533) Nested CTE's with join does not return any data.
[ https://issues.apache.org/jira/browse/HIVE-21533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814945#comment-16814945 ] Naveen Gangam commented on HIVE-21533: -- [~jcamachorodriguez] Do you have any thoughts on what might be the issue? Thank you > Nested CTE's with join does not return any data. > > > Key: HIVE-21533 > URL: https://issues.apache.org/jira/browse/HIVE-21533 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 2.1.0 >Reporter: Naveen Gangam >Priority: Major > Attachments: testcase.sql > > > Attached is the testcase to reproduce the issue. the join on CTE6 is causing > the problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21600) GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS with multiple children
[
https://issues.apache.org/jira/browse/HIVE-21600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814943#comment-16814943
]
Hive QA commented on HIVE-21600:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965512/HIVE-21600.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15896 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=86)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16918/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16918/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16918/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965512 - PreCommit-HIVE-Build
> GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS
> with multiple children
> --
>
> Key: HIVE-21600
> URL: https://issues.apache.org/jira/browse/HIVE-21600
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21600.patch
>
>
> The method does not reset the context when it loops through the children of
> TS. Hence, if TS has multiple FIL children, we can end up with a mangled
> context.
> Stacktrace:
> {code}
> ERROR : FAILED: ArrayIndexOutOfBoundsException -1
> java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:418)
> at java.util.ArrayList.remove(ArrayList.java:495)
> at
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeSemiJoinOperator(GenTezUtils.java:653)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.sharedWorkOptimization(SharedWorkOptimizer.java:452)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.transform(SharedWorkOptimizer.java:151)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:186)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12495)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:362)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:172)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1870)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1817)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1812)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21592) OptimizedSql is not shown when the expression contains CONCAT
[ https://issues.apache.org/jira/browse/HIVE-21592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21592: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master, thanks [~gopalv] > OptimizedSql is not shown when the expression contains CONCAT > - > > Key: HIVE-21592 > URL: https://issues.apache.org/jira/browse/HIVE-21592 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21592.01.patch, HIVE-21592.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19981) Managed tables converted to external tables by the HiveStrictManagedMigration utility should be set to delete data when the table is dropped
[ https://issues.apache.org/jira/browse/HIVE-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814935#comment-16814935 ] Thejas M Nair commented on HIVE-19981: -- Added doc change - https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=27362034=329=327 cc [~kristinehahn] [~leftylev] > Managed tables converted to external tables by the HiveStrictManagedMigration > utility should be set to delete data when the table is dropped > > > Key: HIVE-19981 > URL: https://issues.apache.org/jira/browse/HIVE-19981 > Project: Hive > Issue Type: Sub-task >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-19981.1.patch > > > Using the HiveStrictManagedMigration utility, tables can be converted to > conform to the Hive strict managed tables mode. > For managed tables that are converted to external tables by the utility, > these tables should keep the "drop data on delete" semantics they had when > they were managed tables. > One way to do this is to introduce a table property "external.table.purge", > which if true (and if the table is an external table), will let Hive know to > delete the table data when the table is dropped. This property will be set by > the HiveStrictManagedMigration utility when managed tables are converted to > external tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21600) GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS with multiple children
[
https://issues.apache.org/jira/browse/HIVE-21600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814925#comment-16814925
]
Hive QA commented on HIVE-21600:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
18s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16918/dev-support/hive-personality.sh
|
| git revision | master / be1b39e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16918/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS
> with multiple children
> --
>
> Key: HIVE-21600
> URL: https://issues.apache.org/jira/browse/HIVE-21600
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21600.patch
>
>
> The method does not reset the context when it loops through the children of
> TS. Hence, if TS has multiple FIL children, we can end up with a mangled
> context.
> Stacktrace:
> {code}
> ERROR : FAILED: ArrayIndexOutOfBoundsException -1
> java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:418)
> at java.util.ArrayList.remove(ArrayList.java:495)
> at
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeSemiJoinOperator(GenTezUtils.java:653)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.sharedWorkOptimization(SharedWorkOptimizer.java:452)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.transform(SharedWorkOptimizer.java:151)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:186)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12495)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:362)
> at
>
[jira] [Commented] (HIVE-21500) Disable conversion of managed table to external and vice versa at source.
[
https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814909#comment-16814909
]
Hive QA commented on HIVE-21500:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965496/HIVE-21500.01.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15901 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcWithMiniLlapRow.testComplexQuery (batchId=260)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16917/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16917/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16917/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965496 - PreCommit-HIVE-Build
> Disable conversion of managed table to external and vice versa at source.
> -
>
> Key: HIVE-21500
> URL: https://issues.apache.org/jira/browse/HIVE-21500
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: DR, Replication, pull-request-available
> Attachments: HIVE-21500.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Couple of scenarios for Hive2 to Hive3(strict managed tables enabled)
> replication where managed table is converted to external at source.
> *Scenario-1: (ACID/MM table converted to external at target)*
> 1. Create non-ACID ORC format table.
> 2. Insert some rows
> 3. Replicate this create event which creates ACID table at target (due to
> migration rule). Each insert event adds transactional metadata in HMS
> corresponding to the current table.
> 4. Convert table to external table using ALTER command at source.
> *Scenario-2: (External table at target changes table location)*
> 1. Create non-ACID avro format table.
> 2. Insert some rows
> 3. Replicate this create event which creates external table at target (due to
> migration rule). The data path is chosen under default external warehouse
> directory.
> 4. Convert table to external table using ALTER command at source.
> It is unable to convert an ACID table to external table at target. Also, it
> is hard to detect what would be the table type at target when perform this
> ALTER table operation at source.
> So, it is decided to disable conversion of managed table at source (Hive2) to
> EXTERNAL or vice-versa if the DB is enabled for replication and strict
> managed is disabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21584) Java 11 preparation: system class loader is not URLClassLoader
[
https://issues.apache.org/jira/browse/HIVE-21584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Matyus updated HIVE-21584:
-
Attachment: HIVE-21584.03.patch
> Java 11 preparation: system class loader is not URLClassLoader
> --
>
> Key: HIVE-21584
> URL: https://issues.apache.org/jira/browse/HIVE-21584
> Project: Hive
> Issue Type: Task
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Major
> Attachments: HIVE-21584.01.patch, HIVE-21584.02.patch,
> HIVE-21584.03.patch
>
>
> Currently, Hive assumes that the system class loader is instance of
> {{URLClassLoader}}. In Java 11 this is not the case. There are a few
> (unresolved) JIRAs about specific occurrences of {{URLClassLoader}} (e.g.
> [HIVE-21237|https://issues.apache.org/jira/browse/HIVE-21237],
> [HIVE-17909|https://issues.apache.org/jira/browse/HIVE-17909]), but no
> _"remove all occurrences"_. Also I couldn't find umbrella "Java 11 upgrade"
> JIRA.
> This ticket is to remove all unconditional casts of any random class loader
> to {{URLClassLoader}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21500) Disable conversion of managed table to external and vice versa at source.
[
https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814882#comment-16814882
]
Hive QA commented on HIVE-21500:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
51s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
10s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
19s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 2 new + 49 unchanged - 0 fixed = 51 total (was 49) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
19s{color} | {color:red} itests/hive-unit: The patch generated 4 new + 99
unchanged - 0 fixed = 103 total (was 99) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16917/dev-support/hive-personality.sh
|
| git revision | master / be1b39e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16917/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16917/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: standalone-metastore/metastore-server itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16917/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Disable conversion of managed table to external and vice versa at source.
> -
>
> Key: HIVE-21500
> URL: https://issues.apache.org/jira/browse/HIVE-21500
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: DR, Replication, pull-request-available
> Attachments: HIVE-21500.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Couple of
[jira] [Commented] (HIVE-21593) Break up DDLTask - extract Privilege related operations
[
https://issues.apache.org/jira/browse/HIVE-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814866#comment-16814866
]
Hive QA commented on HIVE-21593:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965480/HIVE-21593.03.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15896 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16916/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16916/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16916/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965480 - PreCommit-HIVE-Build
> Break up DDLTask - extract Privilege related operations
> ---
>
> Key: HIVE-21593
> URL: https://issues.apache.org/jira/browse/HIVE-21593
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-ddl
> Fix For: 4.0.0
>
> Attachments: HIVE-21593.01.patch, HIVE-21593.02.patch,
> HIVE-21593.03.patch
>
>
> DDLTask is a huge class, more than 5000 lines long. The related DDLWork is
> also a huge class, which has a field for each DDL operation it supports. The
> goal is to refactor these in order to have everything cut into more
> handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each operation
> * have a package for each operation group (database ddl, table ddl, etc), so
> the amount of classes under a package is more manageable
> * make all the requests (DDLDesc subclasses) immutable
> * DDLTask should be agnostic to the actual operations
> * right now let's ignore the issue of having some operations handled by
> DDLTask which are not actual DDL operations (lock, unlock, desc...)
> In the interim time when there are two DDLTask and DDLWork classes in the
> code base the new ones in the new package are called DDLTask2 and DDLWork2
> thus avoiding the usage of fully qualified class names where both the old and
> the new classes are in use.
> Step #5: extract all the privilege related operations from the old DDLTask,
> and move them under the new package.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21600) GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS with multiple children
[
https://issues.apache.org/jira/browse/HIVE-21600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-21600:
---
Attachment: HIVE-21600.patch
> GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS
> with multiple children
> --
>
> Key: HIVE-21600
> URL: https://issues.apache.org/jira/browse/HIVE-21600
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21600.patch
>
>
> The method does not reset the context when it loops through the children of
> TS. Hence, if TS has multiple FIL children, we can end up with a mangled
> context.
> Stacktrace:
> {code}
> ERROR : FAILED: ArrayIndexOutOfBoundsException -1
> java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:418)
> at java.util.ArrayList.remove(ArrayList.java:495)
> at
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeSemiJoinOperator(GenTezUtils.java:653)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.sharedWorkOptimization(SharedWorkOptimizer.java:452)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.transform(SharedWorkOptimizer.java:151)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:186)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12495)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:362)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:172)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1870)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1817)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1812)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21600) GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS with multiple children
[
https://issues.apache.org/jira/browse/HIVE-21600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-21600:
---
Status: Patch Available (was: In Progress)
> GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS
> with multiple children
> --
>
> Key: HIVE-21600
> URL: https://issues.apache.org/jira/browse/HIVE-21600
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21600.patch
>
>
> The method does not reset the context when it loops through the children of
> TS. Hence, if TS has multiple FIL children, we can end up with a mangled
> context.
> Stacktrace:
> {code}
> ERROR : FAILED: ArrayIndexOutOfBoundsException -1
> java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:418)
> at java.util.ArrayList.remove(ArrayList.java:495)
> at
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeSemiJoinOperator(GenTezUtils.java:653)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.sharedWorkOptimization(SharedWorkOptimizer.java:452)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.transform(SharedWorkOptimizer.java:151)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:186)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12495)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:362)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:172)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1870)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1817)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1812)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-21600) GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS with multiple children
[
https://issues.apache.org/jira/browse/HIVE-21600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-21600 started by Jesus Camacho Rodriguez.
--
> GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS
> with multiple children
> --
>
> Key: HIVE-21600
> URL: https://issues.apache.org/jira/browse/HIVE-21600
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21600.patch
>
>
> The method does not reset the context when it loops through the children of
> TS. Hence, if TS has multiple FIL children, we can end up with a mangled
> context.
> Stacktrace:
> {code}
> ERROR : FAILED: ArrayIndexOutOfBoundsException -1
> java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:418)
> at java.util.ArrayList.remove(ArrayList.java:495)
> at
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeSemiJoinOperator(GenTezUtils.java:653)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.sharedWorkOptimization(SharedWorkOptimizer.java:452)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.transform(SharedWorkOptimizer.java:151)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:186)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12495)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:362)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:172)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1870)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1817)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1812)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21600) GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS with multiple children
[
https://issues.apache.org/jira/browse/HIVE-21600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-21600:
---
Description:
The method does not reset the context when it loops through the children of TS.
Hence, if TS has multiple FIL children, we can end up with a mangled context.
Stacktrace:
{code}
ERROR : FAILED: ArrayIndexOutOfBoundsException -1
java.lang.ArrayIndexOutOfBoundsException: -1
at java.util.ArrayList.elementData(ArrayList.java:418)
at java.util.ArrayList.remove(ArrayList.java:495)
at
org.apache.hadoop.hive.ql.parse.GenTezUtils.removeSemiJoinOperator(GenTezUtils.java:653)
at
org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.sharedWorkOptimization(SharedWorkOptimizer.java:452)
at
org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.transform(SharedWorkOptimizer.java:151)
at
org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:186)
at
org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12495)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:362)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:172)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1870)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1817)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1812)
{code}
was:The method does not reset the context when it loops through the children
of TS. Hence, if TS has multiple FIL children, we can end up with a mangled
context.
> GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS
> with multiple children
> --
>
> Key: HIVE-21600
> URL: https://issues.apache.org/jira/browse/HIVE-21600
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> The method does not reset the context when it loops through the children of
> TS. Hence, if TS has multiple FIL children, we can end up with a mangled
> context.
> Stacktrace:
> {code}
> ERROR : FAILED: ArrayIndexOutOfBoundsException -1
> java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:418)
> at java.util.ArrayList.remove(ArrayList.java:495)
> at
> org.apache.hadoop.hive.ql.parse.GenTezUtils.removeSemiJoinOperator(GenTezUtils.java:653)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.sharedWorkOptimization(SharedWorkOptimizer.java:452)
> at
> org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer.transform(SharedWorkOptimizer.java:151)
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:186)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12495)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:362)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:172)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1870)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1817)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1812)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21600) GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS with multiple children
[ https://issues.apache.org/jira/browse/HIVE-21600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-21600: -- > GenTezUtils.removeSemiJoinOperator may throw out of bounds exception for TS > with multiple children > -- > > Key: HIVE-21600 > URL: https://issues.apache.org/jira/browse/HIVE-21600 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > The method does not reset the context when it loops through the children of > TS. Hence, if TS has multiple FIL children, we can end up with a mangled > context. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21593) Break up DDLTask - extract Privilege related operations
[
https://issues.apache.org/jira/browse/HIVE-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814847#comment-16814847
]
Hive QA commented on HIVE-21593:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} @author {color} | {color:red} 0m
0s{color} | {color:red} The patch appears to contain 5 @author tags which the
community has agreed to not allow in code contributions. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 2 new + 433 unchanged - 43
fixed = 435 total (was 476) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} ql generated 0 new + 2249 unchanged - 9 fixed = 2249
total (was 2258) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16916/dev-support/hive-personality.sh
|
| git revision | master / be1b39e |
| Default Java | 1.8.0_111 |
| @author |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16916/yetus/author-tags.txt |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16916/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16916/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up DDLTask - extract Privilege related operations
> ---
>
> Key: HIVE-21593
> URL: https://issues.apache.org/jira/browse/HIVE-21593
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-ddl
> Fix For: 4.0.0
>
> Attachments: HIVE-21593.01.patch, HIVE-21593.02.patch,
> HIVE-21593.03.patch
>
>
> DDLTask is a huge class, more than 5000 lines long. The related DDLWork is
> also a huge class, which has a field for each DDL operation it supports. The
> goal is to refactor these in order to have everything cut into more
> handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each operation
> * have a package for each operation group (database ddl, table ddl, etc), so
> the amount of classes under a package is more manageable
> * make all the requests (DDLDesc subclasses) immutable
> * DDLTask should be agnostic to the actual operations
> * right now let's ignore the issue of having some operations handled by
> DDLTask which are not actual DDL operations (lock, unlock, desc...)
> In
[jira] [Comment Edited] (HIVE-21599) Remove predicate on partition columns from Table Scan operator
[ https://issues.apache.org/jira/browse/HIVE-21599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814751#comment-16814751 ] Vineet Garg edited comment on HIVE-21599 at 4/10/19 8:20 PM: - cc [~prasanth_j] [~gopalv] was (Author: vgarg): cc @gopal v[~prasanth_j] > Remove predicate on partition columns from Table Scan operator > -- > > Key: HIVE-21599 > URL: https://issues.apache.org/jira/browse/HIVE-21599 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-21599.1.patch > > > Filter predicates are pushed to Table Scan (to be pushed to and used by > storage handler/input format). Such predicates could consist of partition > columns which are of no use to storage handler or input formats. Therefore > it should be removed from TS filter expression. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-21599) Remove predicate on partition columns from Table Scan operator
[ https://issues.apache.org/jira/browse/HIVE-21599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814751#comment-16814751 ] Vineet Garg edited comment on HIVE-21599 at 4/10/19 8:20 PM: - cc @gopal v[~prasanth_j] was (Author: vgarg): cc [~gopal] [~prasanth_j] > Remove predicate on partition columns from Table Scan operator > -- > > Key: HIVE-21599 > URL: https://issues.apache.org/jira/browse/HIVE-21599 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-21599.1.patch > > > Filter predicates are pushed to Table Scan (to be pushed to and used by > storage handler/input format). Such predicates could consist of partition > columns which are of no use to storage handler or input formats. Therefore > it should be removed from TS filter expression. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21592) OptimizedSql is not shown when the expression contains CONCAT
[
https://issues.apache.org/jira/browse/HIVE-21592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814827#comment-16814827
]
Hive QA commented on HIVE-21592:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965475/HIVE-21592.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15896 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16915/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16915/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16915/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965475 - PreCommit-HIVE-Build
> OptimizedSql is not shown when the expression contains CONCAT
> -
>
> Key: HIVE-21592
> URL: https://issues.apache.org/jira/browse/HIVE-21592
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21592.01.patch, HIVE-21592.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21592) OptimizedSql is not shown when the expression contains CONCAT
[
https://issues.apache.org/jira/browse/HIVE-21592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814805#comment-16814805
]
Hive QA commented on HIVE-21592:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
6s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16915/dev-support/hive-personality.sh
|
| git revision | master / be1b39e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16915/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> OptimizedSql is not shown when the expression contains CONCAT
> -
>
> Key: HIVE-21592
> URL: https://issues.apache.org/jira/browse/HIVE-21592
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21592.01.patch, HIVE-21592.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21584) Java 11 preparation: system class loader is not URLClassLoader
[
https://issues.apache.org/jira/browse/HIVE-21584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814757#comment-16814757
]
Hive QA commented on HIVE-21584:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965435/HIVE-21584.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15902 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16914/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16914/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16914/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965435 - PreCommit-HIVE-Build
> Java 11 preparation: system class loader is not URLClassLoader
> --
>
> Key: HIVE-21584
> URL: https://issues.apache.org/jira/browse/HIVE-21584
> Project: Hive
> Issue Type: Task
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Major
> Attachments: HIVE-21584.01.patch, HIVE-21584.02.patch
>
>
> Currently, Hive assumes that the system class loader is instance of
> {{URLClassLoader}}. In Java 11 this is not the case. There are a few
> (unresolved) JIRAs about specific occurrences of {{URLClassLoader}} (e.g.
> [HIVE-21237|https://issues.apache.org/jira/browse/HIVE-21237],
> [HIVE-17909|https://issues.apache.org/jira/browse/HIVE-17909]), but no
> _"remove all occurrences"_. Also I couldn't find umbrella "Java 11 upgrade"
> JIRA.
> This ticket is to remove all unconditional casts of any random class loader
> to {{URLClassLoader}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21599) Remove predicate on partition columns from Table Scan operator
[ https://issues.apache.org/jira/browse/HIVE-21599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-21599: --- Attachment: HIVE-21599.1.patch > Remove predicate on partition columns from Table Scan operator > -- > > Key: HIVE-21599 > URL: https://issues.apache.org/jira/browse/HIVE-21599 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-21599.1.patch > > > Filter predicates are pushed to Table Scan (to be pushed to and used by > storage handler/input format). Such predicates could consist of partition > columns which are of no use to storage handler or input formats. Therefore > it should be removed from TS filter expression. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21599) Remove predicate on partition columns from Table Scan operator
[ https://issues.apache.org/jira/browse/HIVE-21599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814751#comment-16814751 ] Vineet Garg commented on HIVE-21599: cc [~gopal] [~prasanth_j] > Remove predicate on partition columns from Table Scan operator > -- > > Key: HIVE-21599 > URL: https://issues.apache.org/jira/browse/HIVE-21599 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-21599.1.patch > > > Filter predicates are pushed to Table Scan (to be pushed to and used by > storage handler/input format). Such predicates could consist of partition > columns which are of no use to storage handler or input formats. Therefore > it should be removed from TS filter expression. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21599) Remove predicate on partition columns from Table Scan operator
[ https://issues.apache.org/jira/browse/HIVE-21599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-21599: -- > Remove predicate on partition columns from Table Scan operator > -- > > Key: HIVE-21599 > URL: https://issues.apache.org/jira/browse/HIVE-21599 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > > Filter predicates are pushed to Table Scan (to be pushed to and used by > storage handler/input format). Such predicates could consist of partition > columns which are of no use to storage handler or input formats. Therefore > it should be removed from TS filter expression. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-21540) Query with join condition having date literal throws SemanticException.
[
https://issues.apache.org/jira/browse/HIVE-21540?focusedWorklogId=225761=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-225761
]
ASF GitHub Bot logged work on HIVE-21540:
-
Author: ASF GitHub Bot
Created on: 10/Apr/19 18:21
Start Date: 10/Apr/19 18:21
Worklog Time Spent: 10m
Work Description: sankarh commented on pull request #584: HIVE-21540:
Query with join condition having date literal throws SemanticException.
URL: https://github.com/apache/hive/pull/584
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 225761)
Time Spent: 20m (was: 10m)
> Query with join condition having date literal throws SemanticException.
> ---
>
> Key: HIVE-21540
> URL: https://issues.apache.org/jira/browse/HIVE-21540
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 3.1.0, 3.1.1
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: Analyzer, DateField, pull-request-available
> Fix For: 3.1.0, 4.0.0, 3.1.1
>
> Attachments: HIVE-21540.01.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This semantic exception is thrown for the following query.
> *SemanticException '2019-03-20' encountered with 0 children*
> {code}
> create table date_1 (key int, dd date);
> create table date_2 (key int, dd date);
> select d1.key, d2.dd from(
> select key, dd as start_dd, current_date as end_dd from date_1) d1
> join date_2 as d2 on d1.key = d2.key where d2.dd between start_dd and
> end_dd;
> {code}
> When the WHERE condition below is commented out, the query completes
> successfully.
> where d2.dd between start_dd and end_dd
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21500) Disable conversion of managed table to external and vice versa at source.
[ https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-21500: -- Labels: DR Replication pull-request-available (was: DR Replication) > Disable conversion of managed table to external and vice versa at source. > - > > Key: HIVE-21500 > URL: https://issues.apache.org/jira/browse/HIVE-21500 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication, pull-request-available > Attachments: HIVE-21500.01.patch > > > Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) > replication where managed table is converted to external at source. > *Scenario-1: (ACID/MM table converted to external at target)* > 1. Create non-ACID ORC format table. > 2. Insert some rows > 3. Replicate this create event which creates ACID table at target (due to > migration rule). Each insert event adds transactional metadata in HMS > corresponding to the current table. > 4. Convert table to external table using ALTER command at source. > *Scenario-2: (External table at target changes table location)* > 1. Create non-ACID avro format table. > 2. Insert some rows > 3. Replicate this create event which creates external table at target (due to > migration rule). The data path is chosen under default external warehouse > directory. > 4. Convert table to external table using ALTER command at source. > It is unable to convert an ACID table to external table at target. Also, it > is hard to detect what would be the table type at target when perform this > ALTER table operation at source. > So, it is decided to disable conversion of managed table at source (Hive2) to > EXTERNAL or vice-versa if the DB is enabled for replication and strict > managed is disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21500) Disable conversion of managed table to external and vice versa at source.
[ https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21500: Attachment: HIVE-21500.01.patch > Disable conversion of managed table to external and vice versa at source. > - > > Key: HIVE-21500 > URL: https://issues.apache.org/jira/browse/HIVE-21500 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication > Attachments: HIVE-21500.01.patch > > > Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) > replication where managed table is converted to external at source. > *Scenario-1: (ACID/MM table converted to external at target)* > 1. Create non-ACID ORC format table. > 2. Insert some rows > 3. Replicate this create event which creates ACID table at target (due to > migration rule). Each insert event adds transactional metadata in HMS > corresponding to the current table. > 4. Convert table to external table using ALTER command at source. > *Scenario-2: (External table at target changes table location)* > 1. Create non-ACID avro format table. > 2. Insert some rows > 3. Replicate this create event which creates external table at target (due to > migration rule). The data path is chosen under default external warehouse > directory. > 4. Convert table to external table using ALTER command at source. > It is unable to convert an ACID table to external table at target. Also, it > is hard to detect what would be the table type at target when perform this > ALTER table operation at source. > So, it is decided to disable conversion of managed table at source (Hive2) to > EXTERNAL or vice-versa if the DB is enabled for replication and strict > managed is disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21500) Disable conversion of managed table to external and vice versa at source.
[ https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21500: Status: Patch Available (was: Open) > Disable conversion of managed table to external and vice versa at source. > - > > Key: HIVE-21500 > URL: https://issues.apache.org/jira/browse/HIVE-21500 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication > Attachments: HIVE-21500.01.patch > > > Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) > replication where managed table is converted to external at source. > *Scenario-1: (ACID/MM table converted to external at target)* > 1. Create non-ACID ORC format table. > 2. Insert some rows > 3. Replicate this create event which creates ACID table at target (due to > migration rule). Each insert event adds transactional metadata in HMS > corresponding to the current table. > 4. Convert table to external table using ALTER command at source. > *Scenario-2: (External table at target changes table location)* > 1. Create non-ACID avro format table. > 2. Insert some rows > 3. Replicate this create event which creates external table at target (due to > migration rule). The data path is chosen under default external warehouse > directory. > 4. Convert table to external table using ALTER command at source. > It is unable to convert an ACID table to external table at target. Also, it > is hard to detect what would be the table type at target when perform this > ALTER table operation at source. > So, it is decided to disable conversion of managed table at source (Hive2) to > EXTERNAL or vice-versa if the DB is enabled for replication and strict > managed is disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (HIVE-21500) Disable conversion of managed table to external and vice versa at source.
[ https://issues.apache.org/jira/browse/HIVE-21500?focusedWorklogId=225760=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-225760 ] ASF GitHub Bot logged work on HIVE-21500: - Author: ASF GitHub Bot Created on: 10/Apr/19 18:20 Start Date: 10/Apr/19 18:20 Worklog Time Spent: 10m Work Description: sankarh commented on pull request #591: HIVE-21500: Disable conversion of managed table to external and vice versa at source. URL: https://github.com/apache/hive/pull/591 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 225760) Time Spent: 10m Remaining Estimate: 0h > Disable conversion of managed table to external and vice versa at source. > - > > Key: HIVE-21500 > URL: https://issues.apache.org/jira/browse/HIVE-21500 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication, pull-request-available > Attachments: HIVE-21500.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) > replication where managed table is converted to external at source. > *Scenario-1: (ACID/MM table converted to external at target)* > 1. Create non-ACID ORC format table. > 2. Insert some rows > 3. Replicate this create event which creates ACID table at target (due to > migration rule). Each insert event adds transactional metadata in HMS > corresponding to the current table. > 4. Convert table to external table using ALTER command at source. > *Scenario-2: (External table at target changes table location)* > 1. Create non-ACID avro format table. > 2. Insert some rows > 3. Replicate this create event which creates external table at target (due to > migration rule). The data path is chosen under default external warehouse > directory. > 4. Convert table to external table using ALTER command at source. > It is unable to convert an ACID table to external table at target. Also, it > is hard to detect what would be the table type at target when perform this > ALTER table operation at source. > So, it is decided to disable conversion of managed table at source (Hive2) to > EXTERNAL or vice-versa if the DB is enabled for replication and strict > managed is disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21584) Java 11 preparation: system class loader is not URLClassLoader
[
https://issues.apache.org/jira/browse/HIVE-21584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814740#comment-16814740
]
Hive QA commented on HIVE-21584:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
31s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} spark-client in master has 10 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
43s{color} | {color:blue} llap-server in master has 81 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} beeline in master has 44 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} standalone-metastore/metastore-common: The patch
generated 0 new + 22 unchanged - 2 fixed = 22 total (was 24) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch spark-client passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 1 new + 196 unchanged - 1
fixed = 197 total (was 197) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} llap-server: The patch generated 0 new + 8 unchanged
- 1 fixed = 8 total (was 9) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch beeline passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
40s{color} | {color:green} metastore-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} spark-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} ql generated 0 new + 2254 unchanged - 4 fixed = 2254
total (was 2258) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} llap-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} beeline
[jira] [Updated] (HIVE-21500) Disable conversion of managed table to external and vice versa at source.
[ https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21500: Summary: Disable conversion of managed table to external and vice versa at source. (was: Disable conversion of managed table to external at source.) > Disable conversion of managed table to external and vice versa at source. > - > > Key: HIVE-21500 > URL: https://issues.apache.org/jira/browse/HIVE-21500 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication > > Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) > replication where managed table is converted to external at source. > *Scenario-1: (ACID/MM table converted to external at target)* > 1. Create non-ACID ORC format table. > 2. Insert some rows > 3. Replicate this create event which creates ACID table at target (due to > migration rule). Each insert event adds transactional metadata in HMS > corresponding to the current table. > 4. Convert table to external table using ALTER command at source. > *Scenario-2: (External table at target changes table location)* > 1. Create non-ACID avro format table. > 2. Insert some rows > 3. Replicate this create event which creates external table at target (due to > migration rule). The data path is chosen under default external warehouse > directory. > 4. Convert table to external table using ALTER command at source. > It is unable to convert an ACID table to external table at target. Also, it > is hard to detect what would be the table type at target when perform this > ALTER table operation at source. > So, it is decided to disable conversion of managed table at source (Hive2) to > EXTERNAL or vice-versa if the DB is enabled for replication and strict > managed is disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21597) WM trigger validation should happen at the time of create or alter
[
https://issues.apache.org/jira/browse/HIVE-21597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814677#comment-16814677
]
Hive QA commented on HIVE-21597:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965431/HIVE-21597.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15896 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_rebuild_dummy]
(batchId=166)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16913/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16913/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16913/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965431 - PreCommit-HIVE-Build
> WM trigger validation should happen at the time of create or alter
> --
>
> Key: HIVE-21597
> URL: https://issues.apache.org/jira/browse/HIVE-21597
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-21597.1.patch
>
>
> When a query guardrail trigger is created the trigger expression is not
> validated immediately upon creation or altering the trigger. Instead, it gets
> validated at the start of HS2 which could fail resource plans from being
> applied correctly. The trigger expression validation should happen in
> DDLTask.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21597) WM trigger validation should happen at the time of create or alter
[
https://issues.apache.org/jira/browse/HIVE-21597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814625#comment-16814625
]
Hive QA commented on HIVE-21597:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
54s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
59s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 3 new + 78 unchanged - 0 fixed
= 81 total (was 78) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} service: The patch generated 2 new + 33 unchanged - 2
fixed = 35 total (was 35) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16913/dev-support/hive-personality.sh
|
| git revision | master / be1b39e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16913/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16913/yetus/diff-checkstyle-service.txt
|
| modules | C: ql service U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16913/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> WM trigger validation should happen at the time of create or alter
> --
>
> Key: HIVE-21597
> URL: https://issues.apache.org/jira/browse/HIVE-21597
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-21597.1.patch
>
>
> When a query guardrail trigger is created the trigger expression is not
> validated immediately upon creation or altering the trigger. Instead, it gets
> validated at the start of HS2 which could fail resource plans from being
> applied correctly. The trigger expression validation should happen in
>
[jira] [Updated] (HIVE-21593) Break up DDLTask - extract Privilege related operations
[ https://issues.apache.org/jira/browse/HIVE-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21593: -- Description: DDLTask is a huge class, more than 5000 lines long. The related DDLWork is also a huge class, which has a field for each DDL operation it supports. The goal is to refactor these in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each operation * have a package for each operation group (database ddl, table ddl, etc), so the amount of classes under a package is more manageable * make all the requests (DDLDesc subclasses) immutable * DDLTask should be agnostic to the actual operations * right now let's ignore the issue of having some operations handled by DDLTask which are not actual DDL operations (lock, unlock, desc...) In the interim time when there are two DDLTask and DDLWork classes in the code base the new ones in the new package are called DDLTask2 and DDLWork2 thus avoiding the usage of fully qualified class names where both the old and the new classes are in use. Step #5: extract all the privilege related operations from the old DDLTask, and move them under the new package. was: DDLTask is a huge class, more than 5000 lines long. The related DDLWork is also a huge class, which has a field for each DDL operation it supports. The goal is to refactor these in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each operation * have a package for each operation group (database ddl, table ddl, etc), so the amount of classes under a package is more manageable * make all the requests (DDLDesc subclasses) immutable * DDLTask should be agnostic to the actual operations * right now let's ignore the issue of having some operations handled by DDLTask which are not actual DDL operations (lock, unlock, desc...) In the interim time when there are two DDLTask and DDLWork classes in the code base the new ones in the new package are called DDLTask2 and DDLWork2 thus avoiding the usage of fully qualified class names where both the old and the new classes are in use. Step #4: extract all the privilege related operations from the old DDLTask, and move them under the new package. > Break up DDLTask - extract Privilege related operations > --- > > Key: HIVE-21593 > URL: https://issues.apache.org/jira/browse/HIVE-21593 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21593.01.patch, HIVE-21593.02.patch, > HIVE-21593.03.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #5: extract all the privilege related operations from the old DDLTask, > and move them under the new package. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21593) Break up DDLTask - extract Privilege related operations
[ https://issues.apache.org/jira/browse/HIVE-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21593: -- Attachment: HIVE-21593.03.patch > Break up DDLTask - extract Privilege related operations > --- > > Key: HIVE-21593 > URL: https://issues.apache.org/jira/browse/HIVE-21593 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21593.01.patch, HIVE-21593.02.patch, > HIVE-21593.03.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #4: extract all the privilege related operations from the old DDLTask, > and move them under the new package. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21593) Break up DDLTask - extract Privilege related operations
[ https://issues.apache.org/jira/browse/HIVE-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21593: -- Status: Patch Available (was: Open) > Break up DDLTask - extract Privilege related operations > --- > > Key: HIVE-21593 > URL: https://issues.apache.org/jira/browse/HIVE-21593 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21593.01.patch, HIVE-21593.02.patch, > HIVE-21593.03.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #4: extract all the privilege related operations from the old DDLTask, > and move them under the new package. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21593) Break up DDLTask - extract Privilege related operations
[ https://issues.apache.org/jira/browse/HIVE-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21593: -- Status: Open (was: Patch Available) > Break up DDLTask - extract Privilege related operations > --- > > Key: HIVE-21593 > URL: https://issues.apache.org/jira/browse/HIVE-21593 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21593.01.patch, HIVE-21593.02.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #4: extract all the privilege related operations from the old DDLTask, > and move them under the new package. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21593) Break up DDLTask - extract Privilege related operations
[
https://issues.apache.org/jira/browse/HIVE-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814583#comment-16814583
]
Hive QA commented on HIVE-21593:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965423/HIVE-21593.02.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15896 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=86)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16912/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16912/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16912/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965423 - PreCommit-HIVE-Build
> Break up DDLTask - extract Privilege related operations
> ---
>
> Key: HIVE-21593
> URL: https://issues.apache.org/jira/browse/HIVE-21593
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-ddl
> Fix For: 4.0.0
>
> Attachments: HIVE-21593.01.patch, HIVE-21593.02.patch
>
>
> DDLTask is a huge class, more than 5000 lines long. The related DDLWork is
> also a huge class, which has a field for each DDL operation it supports. The
> goal is to refactor these in order to have everything cut into more
> handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each operation
> * have a package for each operation group (database ddl, table ddl, etc), so
> the amount of classes under a package is more manageable
> * make all the requests (DDLDesc subclasses) immutable
> * DDLTask should be agnostic to the actual operations
> * right now let's ignore the issue of having some operations handled by
> DDLTask which are not actual DDL operations (lock, unlock, desc...)
> In the interim time when there are two DDLTask and DDLWork classes in the
> code base the new ones in the new package are called DDLTask2 and DDLWork2
> thus avoiding the usage of fully qualified class names where both the old and
> the new classes are in use.
> Step #4: extract all the privilege related operations from the old DDLTask,
> and move them under the new package.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21592) OptimizedSql is not shown when the expression contains CONCAT
[ https://issues.apache.org/jira/browse/HIVE-21592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21592: --- Attachment: HIVE-21592.01.patch > OptimizedSql is not shown when the expression contains CONCAT > - > > Key: HIVE-21592 > URL: https://issues.apache.org/jira/browse/HIVE-21592 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21592.01.patch, HIVE-21592.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21568) HiveRelOptUtil.isRowFilteringPlan should skip Project
[ https://issues.apache.org/jira/browse/HIVE-21568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21568: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master, thanks [~vgarg] > HiveRelOptUtil.isRowFilteringPlan should skip Project > - > > Key: HIVE-21568 > URL: https://issues.apache.org/jira/browse/HIVE-21568 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21568.01.patch, HIVE-21568.01.patch, > HIVE-21568.01.patch > > > Project operator should not return true in any case, this may trigger > additional rewritings in presence of constraints. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21593) Break up DDLTask - extract Privilege related operations
[
https://issues.apache.org/jira/browse/HIVE-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814539#comment-16814539
]
Hive QA commented on HIVE-21593:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} @author {color} | {color:red} 0m
0s{color} | {color:red} The patch appears to contain 5 @author tags which the
community has agreed to not allow in code contributions. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 5 new + 433 unchanged - 43
fixed = 438 total (was 476) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
10s{color} | {color:green} ql generated 0 new + 2249 unchanged - 9 fixed = 2249
total (was 2258) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16912/dev-support/hive-personality.sh
|
| git revision | master / 928f3d6 |
| Default Java | 1.8.0_111 |
| @author |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16912/yetus/author-tags.txt |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16912/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16912/yetus/whitespace-eol.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16912/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up DDLTask - extract Privilege related operations
> ---
>
> Key: HIVE-21593
> URL: https://issues.apache.org/jira/browse/HIVE-21593
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-ddl
> Fix For: 4.0.0
>
> Attachments: HIVE-21593.01.patch, HIVE-21593.02.patch
>
>
> DDLTask is a huge class, more than 5000 lines long. The related DDLWork is
> also a huge class, which has a field for each DDL operation it supports. The
> goal is to refactor these in order to have everything cut into more
> handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each operation
> * have a package for each operation group (database ddl, table ddl, etc), so
> the amount of classes under a package is more manageable
> * make all the requests (DDLDesc subclasses) immutable
> * DDLTask should be agnostic to the actual
[jira] [Work logged] (HIVE-21109) Stats replication for ACID tables.
[
https://issues.apache.org/jira/browse/HIVE-21109?focusedWorklogId=225642=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-225642
]
ASF GitHub Bot logged work on HIVE-21109:
-
Author: ASF GitHub Bot
Created on: 10/Apr/19 15:13
Start Date: 10/Apr/19 15:13
Worklog Time Spent: 10m
Work Description: sankarh commented on pull request #579: HIVE-21109 :
Support stats replication for ACID tables.
URL: https://github.com/apache/hive/pull/579#discussion_r273981487
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenarios.java
##
@@ -224,6 +236,17 @@ private void
verifyNoPartitionStatsReplicationForMetadataOnly(String tableName)
}
}
+ private String getCreateTableProperties() {
+if (acidTableKindToUse != null) {
+ if (acidTableKindToUse.equals("orc")) {
Review comment:
To be clear, the name can be "full_acid" and "mm_acid". In fact. MM table
can be created on Orc data.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 225642)
Time Spent: 13h (was: 12h 50m)
> Stats replication for ACID tables.
> --
>
> Key: HIVE-21109
> URL: https://issues.apache.org/jira/browse/HIVE-21109
> Project: Hive
> Issue Type: Sub-task
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21109.01.patch, HIVE-21109.02.patch,
> HIVE-21109.03.patch, HIVE-21109.04.patch, HIVE-21109.05.patch,
> HIVE-21109.06.patch, HIVE-21109.07.patch, HIVE-21109.08.patch,
> HIVE-21109.09.patch, HIVE-21109.09.patch, HIVE-21109.10.patch
>
> Time Spent: 13h
> Remaining Estimate: 0h
>
> Transactional tables require a writeID associated with the stats update. This
> writeId needs to be in sync with the writeId on the source and hence needs to
> be replicated from the source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21109) Stats replication for ACID tables.
[
https://issues.apache.org/jira/browse/HIVE-21109?focusedWorklogId=225640=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-225640
]
ASF GitHub Bot logged work on HIVE-21109:
-
Author: ASF GitHub Bot
Created on: 10/Apr/19 15:13
Start Date: 10/Apr/19 15:13
Worklog Time Spent: 10m
Work Description: sankarh commented on pull request #579: HIVE-21109 :
Support stats replication for ACID tables.
URL: https://github.com/apache/hive/pull/579#discussion_r273984253
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMM.java
##
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.parse;
+
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.metastore.conf.MetastoreConf;
+import
org.apache.hadoop.hive.metastore.messaging.json.gzip.GzipJSONMessageEncoder;
+import org.junit.BeforeClass;
+import org.junit.Rule;
+import org.junit.rules.TestName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Tests statistics replication for ACID tables.
+ */
+public class TestStatsReplicationScenariosMM extends
TestStatsReplicationScenarios {
+ @Rule
+ public final TestName testName = new TestName();
+
+ @BeforeClass
+ public static void classLevelSetup() throws Exception {
+Map overrides = new HashMap<>();
+overrides.put(MetastoreConf.ConfVars.EVENT_MESSAGE_FACTORY.getHiveName(),
+GzipJSONMessageEncoder.class.getCanonicalName());
+overrides.put(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY.varname, "true");
+overrides.put(HiveConf.ConfVars.HIVE_TXN_MANAGER.varname,
+ "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager");
+
overrides.put(MetastoreConf.ConfVars.CAPABILITY_CHECK.getHiveName(),"false");
+
overrides.put(HiveConf.ConfVars.REPL_BOOTSTRAP_DUMP_OPEN_TXN_TIMEOUT.varname,"1s");
+overrides.put(HiveConf.ConfVars.DYNAMICPARTITIONINGMODE.varname,
"nonstrict");
+
+
+internalBeforeClassSetup(overrides, overrides,
TestReplicationScenarios.class, true, "mm");
Review comment:
Shall use current class name TestStatsReplicationScenariosMM.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 225640)
Time Spent: 12h 40m (was: 12.5h)
> Stats replication for ACID tables.
> --
>
> Key: HIVE-21109
> URL: https://issues.apache.org/jira/browse/HIVE-21109
> Project: Hive
> Issue Type: Sub-task
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21109.01.patch, HIVE-21109.02.patch,
> HIVE-21109.03.patch, HIVE-21109.04.patch, HIVE-21109.05.patch,
> HIVE-21109.06.patch, HIVE-21109.07.patch, HIVE-21109.08.patch,
> HIVE-21109.09.patch, HIVE-21109.09.patch, HIVE-21109.10.patch
>
> Time Spent: 12h 40m
> Remaining Estimate: 0h
>
> Transactional tables require a writeID associated with the stats update. This
> writeId needs to be in sync with the writeId on the source and hence needs to
> be replicated from the source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21109) Stats replication for ACID tables.
[
https://issues.apache.org/jira/browse/HIVE-21109?focusedWorklogId=225641=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-225641
]
ASF GitHub Bot logged work on HIVE-21109:
-
Author: ASF GitHub Bot
Created on: 10/Apr/19 15:13
Start Date: 10/Apr/19 15:13
Worklog Time Spent: 10m
Work Description: sankarh commented on pull request #579: HIVE-21109 :
Support stats replication for ACID tables.
URL: https://github.com/apache/hive/pull/579#discussion_r273984105
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestStatsReplicationScenariosMMNoAutogather.java
##
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.parse;
+
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.metastore.conf.MetastoreConf;
+import
org.apache.hadoop.hive.metastore.messaging.json.gzip.GzipJSONMessageEncoder;
+import org.junit.BeforeClass;
+import org.junit.Rule;
+import org.junit.rules.TestName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Tests statistics replication for ACID tables.
+ */
+public class TestStatsReplicationScenariosMMNoAutogather extends
TestStatsReplicationScenarios {
+ @Rule
+ public final TestName testName = new TestName();
+
+ @BeforeClass
+ public static void classLevelSetup() throws Exception {
+Map overrides = new HashMap<>();
+overrides.put(MetastoreConf.ConfVars.EVENT_MESSAGE_FACTORY.getHiveName(),
+GzipJSONMessageEncoder.class.getCanonicalName());
+overrides.put(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY.varname, "true");
+overrides.put(HiveConf.ConfVars.HIVE_TXN_MANAGER.varname,
+ "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager");
+
overrides.put(MetastoreConf.ConfVars.CAPABILITY_CHECK.getHiveName(),"false");
+
overrides.put(HiveConf.ConfVars.REPL_BOOTSTRAP_DUMP_OPEN_TXN_TIMEOUT.varname,"1s");
+overrides.put(HiveConf.ConfVars.DYNAMICPARTITIONINGMODE.varname,
"nonstrict");
+overrides.put("mapred.input.dir.recursive", "true");
+
+
+internalBeforeClassSetup(overrides, overrides,
TestReplicationScenarios.class, false, "mm");
Review comment:
Shall use current class name TestStatsReplicationScenariosMMNoAutogather.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 225641)
Time Spent: 12h 50m (was: 12h 40m)
> Stats replication for ACID tables.
> --
>
> Key: HIVE-21109
> URL: https://issues.apache.org/jira/browse/HIVE-21109
> Project: Hive
> Issue Type: Sub-task
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21109.01.patch, HIVE-21109.02.patch,
> HIVE-21109.03.patch, HIVE-21109.04.patch, HIVE-21109.05.patch,
> HIVE-21109.06.patch, HIVE-21109.07.patch, HIVE-21109.08.patch,
> HIVE-21109.09.patch, HIVE-21109.09.patch, HIVE-21109.10.patch
>
> Time Spent: 12h 50m
> Remaining Estimate: 0h
>
> Transactional tables require a writeID associated with the stats update. This
> writeId needs to be in sync with the writeId on the source and hence needs to
> be replicated from the source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21572) HiveRemoveSqCountCheck rule could be enhanced to capture more patterns
[
https://issues.apache.org/jira/browse/HIVE-21572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814502#comment-16814502
]
Hive QA commented on HIVE-21572:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965398/HIVE-21572.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15896 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar]
(batchId=170)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[mv_query44]
(batchId=275)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query44]
(batchId=275)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16911/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16911/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16911/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965398 - PreCommit-HIVE-Build
> HiveRemoveSqCountCheck rule could be enhanced to capture more patterns
> ---
>
> Key: HIVE-21572
> URL: https://issues.apache.org/jira/browse/HIVE-21572
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-21572.1.patch, HIVE-21572.2.patch,
> HIVE-21572.3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21572) HiveRemoveSqCountCheck rule could be enhanced to capture more patterns
[
https://issues.apache.org/jira/browse/HIVE-21572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814462#comment-16814462
]
Hive QA commented on HIVE-21572:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
14s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 4 new + 1 unchanged - 0 fixed
= 5 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
20s{color} | {color:red} ql generated 1 new + 2258 unchanged - 0 fixed = 2259
total (was 2258) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to filter in
org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRemoveSqCountCheck.onMatch(RelOptRuleCall)
At
HiveRemoveSqCountCheck.java:org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRemoveSqCountCheck.onMatch(RelOptRuleCall)
At HiveRemoveSqCountCheck.java:[line 134] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16911/dev-support/hive-personality.sh
|
| git revision | master / 928f3d6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16911/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16911/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16911/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HiveRemoveSqCountCheck rule could be enhanced to capture more patterns
> ---
>
> Key: HIVE-21572
> URL: https://issues.apache.org/jira/browse/HIVE-21572
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-21572.1.patch, HIVE-21572.2.patch,
> HIVE-21572.3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[
https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814397#comment-16814397
]
Hive QA commented on HIVE-13582:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965380/HIVE-13582.1.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 183 failed/errored test(s), 15885 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
[intersect_all.q,unionDistinct_1.q,table_nonprintable.q,orc_llap_counters1.q,mm_cttas.q,whroot_external1.q,global_limit.q,cte_2.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,results_cache_diff_fs.q,cttl.q,parallel_colstats.q,load_hdfs_file_with_space_in_the_name.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bool_unknown]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_subq_exists]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constant_prop_3]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constprog_partitioner]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_1] (batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[keep_uniform]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_notexists]
(batchId=98)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_notexists_having]
(batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_notin_having]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_unqualcolumnrefs]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_groupby_mapjoin]
(batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_mapjoin3]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[view_cbo] (batchId=74)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_subq_exists]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_subq_in]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_subq_not_in]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cte_1]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[external_jdbc_table_perf]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_groupingset_bug]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[keep_uniform]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage3]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mapjoin_hint]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_join_ptp]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_ALL]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_ANY]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in_having]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_multi]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_notin]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_null_agg]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_views]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_reduce_side]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_mapjoin]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_mapjoin_reduce]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_mapjoin3]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[constprog_partitioner]
(batchId=192)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=191)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[invalid_select_expression]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_in_lhs]
(batchId=99)
[jira] [Commented] (HIVE-13582) E061-07 and E061-12: Quantified Comparison Predicates
[
https://issues.apache.org/jira/browse/HIVE-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814357#comment-16814357
]
Hive QA commented on HIVE-13582:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
2s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 97 new + 480 unchanged - 7
fixed = 577 total (was 487) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 10 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 8 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
17s{color} | {color:red} ql generated 14 new + 2249 unchanged - 9 fixed = 2263
total (was 2258) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Switch statement found in
org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveSubQueryRemoveRule.rewriteInExists(RexSubQuery,
Set, RelOptUtil$Logic, HiveSubQRemoveRelBuilder, int, boolean) where one case
falls through to the next case At HiveSubQueryRemoveRule.java:Set,
RelOptUtil$Logic, HiveSubQRemoveRelBuilder, int, boolean) where one case falls
through to the next case At HiveSubQueryRemoveRule.java:[lines 459-462] |
| | Switch statement found in
org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveSubQueryRemoveRule.rewriteInExists(RexSubQuery,
Set, RelOptUtil$Logic, HiveSubQRemoveRelBuilder, int, boolean) where default
case is missing At HiveSubQueryRemoveRule.java:Set, RelOptUtil$Logic,
HiveSubQRemoveRelBuilder, int, boolean) where default case is missing At
HiveSubQueryRemoveRule.java:[lines 355-377] |
| | Dead store to stream_retval in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceSimilarExpressionQuantifierPredicate(CommonTree)
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceSimilarExpressionQuantifierPredicate(CommonTree)
At HiveParser_IdentifiersParser.java:[line 9822] |
| | Dead store to stream_retval in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.quantifierType()
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.quantifierType()
At HiveParser_IdentifiersParser.java:[line 9940] |
| | Redundant nullcheck of nonReserved311, which is known to be non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier()
Redundant null check at HiveParser_IdentifiersParser.java:is known to be
[jira] [Commented] (HIVE-21531) Vectorization: all NULL hashcodes are not computed using Murmur3
[
https://issues.apache.org/jira/browse/HIVE-21531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814323#comment-16814323
]
Hive QA commented on HIVE-21531:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965383/HIVE-21531.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15898 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_limit]
(batchId=173)
org.apache.hive.hcatalog.mapreduce.TestHCatMutableNonPartitioned.testHCatNonPartitionedTable[1]
(batchId=214)
org.apache.hive.hcatalog.mapreduce.TestHCatMutableNonPartitioned.testHCatNonPartitionedTable[3]
(batchId=214)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16909/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16909/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16909/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965383 - PreCommit-HIVE-Build
> Vectorization: all NULL hashcodes are not computed using Murmur3
>
>
> Key: HIVE-21531
> URL: https://issues.apache.org/jira/browse/HIVE-21531
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0, 3.1.1
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Critical
> Attachments: HIVE-21531.1.patch, HIVE-21531.1.patch,
> HIVE-21531.WIP.patch
>
>
> The comments in Vectorized hash computation call out the MurmurHash
> implementation (the one using 0x5bd1e995), while the non-vectorized codepath
> calls out the Murmur3 one (using 0xcc9e2d51).
> The comments here are wrong
> {code}
> /**
>* Batch compute the hash codes for all the serialized keys.
>*
>* NOTE: MAJOR MAJOR ASSUMPTION:
>* We assume that HashCodeUtil.murmurHash produces the same result
>* as MurmurHash.hash with seed = 0 (the method used by
> ReduceSinkOperator for
>* UNIFORM distribution).
>*/
> protected void computeSerializedHashCodes() {
> int offset = 0;
> int keyLength;
> byte[] bytes = output.getData();
> for (int i = 0; i < nonNullKeyCount; i++) {
> keyLength = serializedKeyLengths[i];
> hashCodes[i] = Murmur3.hash32(bytes, offset, keyLength, 0);
> offset += keyLength;
> }
> }
> {code}
> but the wrong comment is followed in the Vector RS operator
> {code}
> System.arraycopy(nullKeyOutput.getData(), 0, nullBytes, 0,
> nullBytesLength);
> nullKeyHashCode = HashCodeUtil.calculateBytesHashCode(nullBytes, 0,
> nullBytesLength);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21531) Vectorization: all NULL hashcodes are not computed using Murmur3
[
https://issues.apache.org/jira/browse/HIVE-21531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814288#comment-16814288
]
Hive QA commented on HIVE-21531:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
8s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 10 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16909/dev-support/hive-personality.sh
|
| git revision | master / 928f3d6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16909/yetus/whitespace-eol.txt
|
| modules | C: common ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16909/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: all NULL hashcodes are not computed using Murmur3
>
>
> Key: HIVE-21531
> URL: https://issues.apache.org/jira/browse/HIVE-21531
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0, 3.1.1
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Critical
> Attachments: HIVE-21531.1.patch, HIVE-21531.1.patch,
> HIVE-21531.WIP.patch
>
>
> The comments in Vectorized hash computation call out the MurmurHash
> implementation (the one using 0x5bd1e995), while the non-vectorized codepath
> calls out the Murmur3 one (using 0xcc9e2d51).
> The comments here are wrong
> {code}
> /**
>* Batch compute the hash codes for all the serialized keys.
>*
>* NOTE: MAJOR MAJOR ASSUMPTION:
>* We assume that HashCodeUtil.murmurHash produces the same result
>* as MurmurHash.hash with seed = 0 (the method used by
> ReduceSinkOperator for
>* UNIFORM distribution).
>
[jira] [Commented] (HIVE-21584) Java 11 preparation: system class loader is not URLClassLoader
[
https://issues.apache.org/jira/browse/HIVE-21584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814274#comment-16814274
]
Zoltan Matyus commented on HIVE-21584:
--
bq. org.apache.hadoop.hive.ql.exec.Utilities.createUDFClassLoader(ClassLoader,
String[]) creates a org.apache.hadoop.hive.ql.exec.UDFClassLoader classloader,
which should be performed within a doPrivileged block At
Utilities.java:classloader, which should be performed within a doPrivileged
block At Utilities.java:[line 2126]
Creating class loader is not new functionality, but now that I modified the
code, FindBugs complains. This could be solved fairly easily, by hiding the
{{AccessController.doPrivileged}} call in the CreateUDFClassLoader method.
However, Oracle [strongly
recommends|https://docs.oracle.com/javase/8/docs/technotes/guides/security/doprivileged.html]
not to do this:
bq. Do not be tempted to write a utility class that itself calls doPrivileged
as that could lead to security holes. You can write utility classes for
PrivilegedAction classes though, as shown in the preceding example.
I followed this approach, which meant a small change at each caller's location.
Also, I merged the addToClassPath and the createNewClassLoader functionality
into an AddToClassPath action with a forceNewClassLoader flag...
> Java 11 preparation: system class loader is not URLClassLoader
> --
>
> Key: HIVE-21584
> URL: https://issues.apache.org/jira/browse/HIVE-21584
> Project: Hive
> Issue Type: Task
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Major
> Attachments: HIVE-21584.01.patch, HIVE-21584.02.patch
>
>
> Currently, Hive assumes that the system class loader is instance of
> {{URLClassLoader}}. In Java 11 this is not the case. There are a few
> (unresolved) JIRAs about specific occurrences of {{URLClassLoader}} (e.g.
> [HIVE-21237|https://issues.apache.org/jira/browse/HIVE-21237],
> [HIVE-17909|https://issues.apache.org/jira/browse/HIVE-17909]), but no
> _"remove all occurrences"_. Also I couldn't find umbrella "Java 11 upgrade"
> JIRA.
> This ticket is to remove all unconditional casts of any random class loader
> to {{URLClassLoader}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21427) Syslog storage handler
[
https://issues.apache.org/jira/browse/HIVE-21427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814266#comment-16814266
]
Hive QA commented on HIVE-21427:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
59s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
12s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} llap-server in master has 81 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/util in master has 46 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 5 new + 38 unchanged - 0 fixed = 43 total (was 38) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 172 new + 93 unchanged - 0
fixed = 265 total (was 93) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
3s{color} | {color:red} root: The patch generated 178 new + 653 unchanged - 0
fixed = 831 total (was 653) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} itests/util: The patch generated 1 new + 42 unchanged
- 0 fixed = 43 total (was 42) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 11 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 9 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
31s{color} | {color:red} patch/common cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
9s{color} | {color:red} patch/standalone-metastore/metastore-server cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
59s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
53s{color} | {color:red} patch/llap-server cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
41s{color} | {color:red} patch/itests/util cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m
43s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 8m
40s{color}
[jira] [Updated] (HIVE-21584) Java 11 preparation: system class loader is not URLClassLoader
[
https://issues.apache.org/jira/browse/HIVE-21584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Matyus updated HIVE-21584:
-
Attachment: HIVE-21584.02.patch
> Java 11 preparation: system class loader is not URLClassLoader
> --
>
> Key: HIVE-21584
> URL: https://issues.apache.org/jira/browse/HIVE-21584
> Project: Hive
> Issue Type: Task
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Major
> Attachments: HIVE-21584.01.patch, HIVE-21584.02.patch
>
>
> Currently, Hive assumes that the system class loader is instance of
> {{URLClassLoader}}. In Java 11 this is not the case. There are a few
> (unresolved) JIRAs about specific occurrences of {{URLClassLoader}} (e.g.
> [HIVE-21237|https://issues.apache.org/jira/browse/HIVE-21237],
> [HIVE-17909|https://issues.apache.org/jira/browse/HIVE-17909]), but no
> _"remove all occurrences"_. Also I couldn't find umbrella "Java 11 upgrade"
> JIRA.
> This ticket is to remove all unconditional casts of any random class loader
> to {{URLClassLoader}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21427) Syslog storage handler
[
https://issues.apache.org/jira/browse/HIVE-21427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814249#comment-16814249
]
Hive QA commented on HIVE-21427:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965301/HIVE-21427.8.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15908 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16908/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16908/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16908/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965301 - PreCommit-HIVE-Build
> Syslog storage handler
> --
>
> Key: HIVE-21427
> URL: https://issues.apache.org/jira/browse/HIVE-21427
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
>Affects Versions: 4.0.0
>Reporter: Ashutosh Chauhan
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21427.1.patch, HIVE-21427.2.patch,
> HIVE-21427.3.patch, HIVE-21427.4.patch, HIVE-21427.5.patch,
> HIVE-21427.6.patch, HIVE-21427.7.patch, HIVE-21427.8.patch
>
>
> It will be useful to read syslog generated log files in Hive. Hive generates
> logs in RFC5424 log4j2 layout and stores it as external table in sys.db. This
> includes a SyslogSerde that can parse RFC5424 formatted logs and maps them to
> logs table schema for query processing by hive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21597) WM trigger validation should happen at the time of create or alter
[ https://issues.apache.org/jira/browse/HIVE-21597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814231#comment-16814231 ] Prasanth Jayachandran commented on HIVE-21597: -- 3 small changes * Validates trigger during create/alter * Includes log message for HIVE-21594 * Improved the exception error message when kill query fails [~daijy] can you please review since you have more context on the issues? > WM trigger validation should happen at the time of create or alter > -- > > Key: HIVE-21597 > URL: https://issues.apache.org/jira/browse/HIVE-21597 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-21597.1.patch > > > When a query guardrail trigger is created the trigger expression is not > validated immediately upon creation or altering the trigger. Instead, it gets > validated at the start of HS2 which could fail resource plans from being > applied correctly. The trigger expression validation should happen in > DDLTask. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21597) WM trigger validation should happen at the time of create or alter
[ https://issues.apache.org/jira/browse/HIVE-21597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21597: - Attachment: HIVE-21597.1.patch > WM trigger validation should happen at the time of create or alter > -- > > Key: HIVE-21597 > URL: https://issues.apache.org/jira/browse/HIVE-21597 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-21597.1.patch > > > When a query guardrail trigger is created the trigger expression is not > validated immediately upon creation or altering the trigger. Instead, it gets > validated at the start of HS2 which could fail resource plans from being > applied correctly. The trigger expression validation should happen in > DDLTask. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21597) WM trigger validation should happen at the time of create or alter
[ https://issues.apache.org/jira/browse/HIVE-21597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21597: - Status: Patch Available (was: Open) > WM trigger validation should happen at the time of create or alter > -- > > Key: HIVE-21597 > URL: https://issues.apache.org/jira/browse/HIVE-21597 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-21597.1.patch > > > When a query guardrail trigger is created the trigger expression is not > validated immediately upon creation or altering the trigger. Instead, it gets > validated at the start of HS2 which could fail resource plans from being > applied correctly. The trigger expression validation should happen in > DDLTask. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21594) Warning regarding multiple MOVE triggers is logged even if conflicting triggers are both KILL
[
https://issues.apache.org/jira/browse/HIVE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814228#comment-16814228
]
Prasanth Jayachandran commented on HIVE-21594:
--
[~bgoerlitz] I will cover this in HIVE-21597 since this is logging only change.
> Warning regarding multiple MOVE triggers is logged even if conflicting
> triggers are both KILL
> -
>
> Key: HIVE-21594
> URL: https://issues.apache.org/jira/browse/HIVE-21594
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Brian Goerlitz
>Assignee: Prasanth Jayachandran
>Priority: Minor
>
> The following in TriggerValidatorRunnable will log a WARN about conflicting
> MOVE triggers even if the previous trigger violated was a KILL trigger.
>
> {code:java}
> } else {
> // if multiple MOVE happens, only first move will be chosen
> LOG.warn("Conflicting MOVE triggers ({} and {}). Choosing the first MOVE
> trigger: {}", existingTrigger, currentTrigger, existingTrigger.getName());
> {code}
>
> This logging makes sense if the code broke out of the triggers loop on the
> first encountered KILL trigger violation, but this currently does not happen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21570) Convert llap iomem servlets output to json format
[ https://issues.apache.org/jira/browse/HIVE-21570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814222#comment-16814222 ] Antal Sinkovits commented on HIVE-21570: [~bslim] [~szita] [~draese] Could you please review the change. Thanks. > Convert llap iomem servlets output to json format > - > > Key: HIVE-21570 > URL: https://issues.apache.org/jira/browse/HIVE-21570 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Antal Sinkovits >Assignee: Antal Sinkovits >Priority: Minor > Attachments: HIVE-21570.01.patch, HIVE-21570.02.patch, > HIVE-21570.03.patch, HIVE-21570.04.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21109) Stats replication for ACID tables.
[
https://issues.apache.org/jira/browse/HIVE-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814175#comment-16814175
]
Hive QA commented on HIVE-21109:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965356/HIVE-21109.10.patch
{color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 15861 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=254)
TestLocationQueries - did not produce a TEST-*.xml file (likely timed out)
(batchId=254)
TestReplAcrossInstancesWithJsonMessageFormat - did not produce a TEST-*.xml
file (likely timed out) (batchId=247)
TestReplicationOnHDFSEncryptedZones - did not produce a TEST-*.xml file (likely
timed out) (batchId=247)
TestReplicationScenariosExternalTables - did not produce a TEST-*.xml file
(likely timed out) (batchId=247)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=254)
TestStatsReplicationScenarios - did not produce a TEST-*.xml file (likely timed
out) (batchId=254)
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=247)
TestStatsReplicationScenariosACIDNoAutogather - did not produce a TEST-*.xml
file (likely timed out) (batchId=247)
TestStatsReplicationScenariosMM - did not produce a TEST-*.xml file (likely
timed out) (batchId=247)
TestStatsReplicationScenariosMigration - did not produce a TEST-*.xml file
(likely timed out) (batchId=247)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=86)
org.apache.hadoop.hive.ql.exec.tez.TestDynamicPartitionPruner.testSingleSourceMultipleFiltersOrdering1
(batchId=320)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16907/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16907/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16907/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965356 - PreCommit-HIVE-Build
> Stats replication for ACID tables.
> --
>
> Key: HIVE-21109
> URL: https://issues.apache.org/jira/browse/HIVE-21109
> Project: Hive
> Issue Type: Sub-task
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21109.01.patch, HIVE-21109.02.patch,
> HIVE-21109.03.patch, HIVE-21109.04.patch, HIVE-21109.05.patch,
> HIVE-21109.06.patch, HIVE-21109.07.patch, HIVE-21109.08.patch,
> HIVE-21109.09.patch, HIVE-21109.09.patch, HIVE-21109.10.patch
>
> Time Spent: 12.5h
> Remaining Estimate: 0h
>
> Transactional tables require a writeID associated with the stats update. This
> writeId needs to be in sync with the writeId on the source and hence needs to
> be replicated from the source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21593) Break up DDLTask - extract Privilege related operations
[ https://issues.apache.org/jira/browse/HIVE-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21593: -- Attachment: HIVE-21593.02.patch > Break up DDLTask - extract Privilege related operations > --- > > Key: HIVE-21593 > URL: https://issues.apache.org/jira/browse/HIVE-21593 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21593.01.patch, HIVE-21593.02.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #4: extract all the privilege related operations from the old DDLTask, > and move them under the new package. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21593) Break up DDLTask - extract Privilege related operations
[ https://issues.apache.org/jira/browse/HIVE-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21593: -- Status: Patch Available (was: Open) > Break up DDLTask - extract Privilege related operations > --- > > Key: HIVE-21593 > URL: https://issues.apache.org/jira/browse/HIVE-21593 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21593.01.patch, HIVE-21593.02.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #4: extract all the privilege related operations from the old DDLTask, > and move them under the new package. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21593) Break up DDLTask - extract Privilege related operations
[ https://issues.apache.org/jira/browse/HIVE-21593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21593: -- Status: Open (was: Patch Available) > Break up DDLTask - extract Privilege related operations > --- > > Key: HIVE-21593 > URL: https://issues.apache.org/jira/browse/HIVE-21593 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-21593.01.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #4: extract all the privilege related operations from the old DDLTask, > and move them under the new package. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21109) Stats replication for ACID tables.
[
https://issues.apache.org/jira/browse/HIVE-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814150#comment-16814150
]
Hive QA commented on HIVE-21109:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
11s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
6s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
28s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 2 new + 952 unchanged - 0 fixed = 954 total (was 952) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
51s{color} | {color:red} ql: The patch generated 30 new + 677 unchanged - 3
fixed = 707 total (was 680) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
19s{color} | {color:red} itests/hive-unit: The patch generated 152 new + 4
unchanged - 0 fixed = 156 total (was 4) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
21s{color} | {color:red} ql generated 3 new + 2257 unchanged - 1 fixed = 2260
total (was 2258) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 27s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Suspicious comparison of a Long reference to constant in
org.apache.hadoop.hive.ql.exec.ReplCopyTask.execute(DriverContext) At
ReplCopyTask.java:reference to constant in
org.apache.hadoop.hive.ql.exec.ReplCopyTask.execute(DriverContext) At
ReplCopyTask.java:[line 216] |
| | Exception is caught when Exception is not thrown in
org.apache.hadoop.hive.ql.exec.repl.bootstrap.events.filesystem.FSTableEvent.partitionDesc(Path,
ImportTableDesc, Partition) At FSTableEvent.java:is not thrown in
org.apache.hadoop.hive.ql.exec.repl.bootstrap.events.filesystem.FSTableEvent.partitionDesc(Path,
ImportTableDesc, Partition) At FSTableEvent.java:[line 210] |
| | Redundant nullcheck of val, which is known to be non-null in
org.apache.hadoop.hive.ql.metadata.Hive.renamePartition(Table, Map, Partition,
long) Redundant null check at Hive.java:is known to be non-null in
org.apache.hadoop.hive.ql.metadata.Hive.renamePartition(Table, Map, Partition,
long) Redundant null check at Hive.java:[line 901] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux
[jira] [Assigned] (HIVE-21597) WM trigger validation should happen at the time of create or alter
[ https://issues.apache.org/jira/browse/HIVE-21597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-21597: > WM trigger validation should happen at the time of create or alter > -- > > Key: HIVE-21597 > URL: https://issues.apache.org/jira/browse/HIVE-21597 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0, 3.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > > When a query guardrail trigger is created the trigger expression is not > validated immediately upon creation or altering the trigger. Instead, it gets > validated at the start of HS2 which could fail resource plans from being > applied correctly. The trigger expression validation should happen in > DDLTask. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21594) Warning regarding multiple MOVE triggers is logged even if conflicting triggers are both KILL
[
https://issues.apache.org/jira/browse/HIVE-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran reassigned HIVE-21594:
Assignee: Prasanth Jayachandran
> Warning regarding multiple MOVE triggers is logged even if conflicting
> triggers are both KILL
> -
>
> Key: HIVE-21594
> URL: https://issues.apache.org/jira/browse/HIVE-21594
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Brian Goerlitz
>Assignee: Prasanth Jayachandran
>Priority: Minor
>
> The following in TriggerValidatorRunnable will log a WARN about conflicting
> MOVE triggers even if the previous trigger violated was a KILL trigger.
>
> {code:java}
> } else {
> // if multiple MOVE happens, only first move will be chosen
> LOG.warn("Conflicting MOVE triggers ({} and {}). Choosing the first MOVE
> trigger: {}", existingTrigger, currentTrigger, existingTrigger.getName());
> {code}
>
> This logging makes sense if the code broke out of the triggers loop on the
> first encountered KILL trigger violation, but this currently does not happen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21568) HiveRelOptUtil.isRowFilteringPlan should skip Project
[
https://issues.apache.org/jira/browse/HIVE-21568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814124#comment-16814124
]
Hive QA commented on HIVE-21568:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12965357/HIVE-21568.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15896 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16906/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16906/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16906/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12965357 - PreCommit-HIVE-Build
> HiveRelOptUtil.isRowFilteringPlan should skip Project
> -
>
> Key: HIVE-21568
> URL: https://issues.apache.org/jira/browse/HIVE-21568
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21568.01.patch, HIVE-21568.01.patch,
> HIVE-21568.01.patch
>
>
> Project operator should not return true in any case, this may trigger
> additional rewritings in presence of constraints.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21568) HiveRelOptUtil.isRowFilteringPlan should skip Project
[
https://issues.apache.org/jira/browse/HIVE-21568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814082#comment-16814082
]
Hive QA commented on HIVE-21568:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 1 new + 15 unchanged - 0 fixed
= 16 total (was 15) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16906/dev-support/hive-personality.sh
|
| git revision | master / 928f3d6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16906/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16906/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HiveRelOptUtil.isRowFilteringPlan should skip Project
> -
>
> Key: HIVE-21568
> URL: https://issues.apache.org/jira/browse/HIVE-21568
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21568.01.patch, HIVE-21568.01.patch,
> HIVE-21568.01.patch
>
>
> Project operator should not return true in any case, this may trigger
> additional rewritings in presence of constraints.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21500) Disable conversion of managed table to external at source.
[ https://issues.apache.org/jira/browse/HIVE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21500: Description: Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) replication where managed table is converted to external at source. *Scenario-1: (ACID/MM table converted to external at target)* 1. Create non-ACID ORC format table. 2. Insert some rows 3. Replicate this create event which creates ACID table at target (due to migration rule). Each insert event adds transactional metadata in HMS corresponding to the current table. 4. Convert table to external table using ALTER command at source. *Scenario-2: (External table at target changes table location)* 1. Create non-ACID avro format table. 2. Insert some rows 3. Replicate this create event which creates external table at target (due to migration rule). The data path is chosen under default external warehouse directory. 4. Convert table to external table using ALTER command at source. It is unable to convert an ACID table to external table at target. Also, it is hard to detect what would be the table type at target when perform this ALTER table operation at source. So, it is decided to disable conversion of managed table at source (Hive2) to EXTERNAL or vice-versa if the DB is enabled for replication and strict managed is disabled. was: Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) replication where managed table is converted to external at source. *Scenario-1: (ACID/MM table converted to external at target)* 1. Create non-ACID ORC format table. 2. Insert some rows 3. Replicate this create event which creates ACID table at target (due to migration rule). Each insert event adds transactional metadata in HMS corresponding to the current table. 4. Convert table to external table using ALTER command at source. *Scenario-2: (External table at target changes table location)* 1. Create non-ACID avro format table. 2. Insert some rows 3. Replicate this create event which creates external table at target (due to migration rule). The data path is chosen under default external warehouse directory. 4. Convert table to external table using ALTER command at source. It is unable to convert an ACID table to external table at target. Also, it is hard to detect what would be the table type at target when perform this ALTER table operation at source. So, it is decided to disable conversion of managed table at source (Hive2) to EXTERNAL or vice-versa. > Disable conversion of managed table to external at source. > -- > > Key: HIVE-21500 > URL: https://issues.apache.org/jira/browse/HIVE-21500 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication > > Couple of scenarios for Hive2 to Hive3(strict managed tables enabled) > replication where managed table is converted to external at source. > *Scenario-1: (ACID/MM table converted to external at target)* > 1. Create non-ACID ORC format table. > 2. Insert some rows > 3. Replicate this create event which creates ACID table at target (due to > migration rule). Each insert event adds transactional metadata in HMS > corresponding to the current table. > 4. Convert table to external table using ALTER command at source. > *Scenario-2: (External table at target changes table location)* > 1. Create non-ACID avro format table. > 2. Insert some rows > 3. Replicate this create event which creates external table at target (due to > migration rule). The data path is chosen under default external warehouse > directory. > 4. Convert table to external table using ALTER command at source. > It is unable to convert an ACID table to external table at target. Also, it > is hard to detect what would be the table type at target when perform this > ALTER table operation at source. > So, it is decided to disable conversion of managed table at source (Hive2) to > EXTERNAL or vice-versa if the DB is enabled for replication and strict > managed is disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
