[jira] [Commented] (HIVE-22307) Upgrade Hadoop version to 3.1.3
[ https://issues.apache.org/jira/browse/HIVE-22307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17226564#comment-17226564 ] Viacheslav Avramenko commented on HIVE-22307: - [~dlavati], [~kgyrtkirk] Druid currently depends on guava 16, which makes it incompatible with hadoop 3.1.3. Hence tests in hive for druid is failing. Not sure how to fix it. > Upgrade Hadoop version to 3.1.3 > --- > > Key: HIVE-22307 > URL: https://issues.apache.org/jira/browse/HIVE-22307 > Project: Hive > Issue Type: Improvement >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Attachments: HIVE-22307.patch, HIVE-22307.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22307) Upgrade Hadoop version to 3.1.3
[ https://issues.apache.org/jira/browse/HIVE-22307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225735#comment-17225735 ] Viacheslav Avramenko commented on HIVE-22307: - thanks for a hint [~kgyrtkirk], I've changed to lambda instead because spotbugs wasn't happy even after I marked method and parameter with nullable > Upgrade Hadoop version to 3.1.3 > --- > > Key: HIVE-22307 > URL: https://issues.apache.org/jira/browse/HIVE-22307 > Project: Hive > Issue Type: Improvement >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Attachments: HIVE-22307.patch, HIVE-22307.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22307) Upgrade Hadoop version to 3.1.3
[ https://issues.apache.org/jira/browse/HIVE-22307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224236#comment-17224236 ] Viacheslav Avramenko commented on HIVE-22307: - [~dlavati] FYI PR for upgrade guava to 27.0-jre and hadoop - https://github.com/apache/hive/pull/1638 There is unrelated build failure thou: row must be non-null but is marked as nullable [org.apache.hadoop.hive.common.log.InPlaceUpdate$1] At InPlaceUpdate.java:[line 164] NP_PARAMETER_MUST_BE_NONNULL_BUT_MARKED_AS_NULLABLE > Upgrade Hadoop version to 3.1.3 > --- > > Key: HIVE-22307 > URL: https://issues.apache.org/jira/browse/HIVE-22307 > Project: Hive > Issue Type: Improvement >Reporter: David Lavati >Assignee: Viacheslav Avramenko >Priority: Major > Attachments: HIVE-22307.patch, HIVE-22307.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22307) Upgrade Hadoop version to 3.1.3
[ https://issues.apache.org/jira/browse/HIVE-22307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viacheslav Avramenko reassigned HIVE-22307: --- Assignee: David Lavati (was: Viacheslav Avramenko) > Upgrade Hadoop version to 3.1.3 > --- > > Key: HIVE-22307 > URL: https://issues.apache.org/jira/browse/HIVE-22307 > Project: Hive > Issue Type: Improvement >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Attachments: HIVE-22307.patch, HIVE-22307.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22307) Upgrade Hadoop version to 3.1.3
[ https://issues.apache.org/jira/browse/HIVE-22307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viacheslav Avramenko reassigned HIVE-22307: --- Assignee: Viacheslav Avramenko (was: David Lavati) > Upgrade Hadoop version to 3.1.3 > --- > > Key: HIVE-22307 > URL: https://issues.apache.org/jira/browse/HIVE-22307 > Project: Hive > Issue Type: Improvement >Reporter: David Lavati >Assignee: Viacheslav Avramenko >Priority: Major > Attachments: HIVE-22307.patch, HIVE-22307.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22912) Support native submission of Hive queries to a Kubernetes Cluster
[ https://issues.apache.org/jira/browse/HIVE-22912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17219943#comment-17219943 ] Viacheslav Avramenko commented on HIVE-22912: - I agreed with Surbhi and Michel, what about the kubernetes support as open-source project? > Support native submission of Hive queries to a Kubernetes Cluster > - > > Key: HIVE-22912 > URL: https://issues.apache.org/jira/browse/HIVE-22912 > Project: Hive > Issue Type: New Feature >Reporter: Surbhi Aggarwal >Priority: Major > > So many big data applications are already integrated or trying to natively > integrate with Kubernetes engine. Should we not work together to support hive > with this engine? > If efforts are already being spent on this, please point me to it. Thanks ! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24256) REPL LOAD fails because of unquoted column name
[
https://issues.apache.org/jira/browse/HIVE-24256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viacheslav Avramenko updated HIVE-24256:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> REPL LOAD fails because of unquoted column name
> ---
>
> Key: HIVE-24256
> URL: https://issues.apache.org/jira/browse/HIVE-24256
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Viacheslav Avramenko
>Assignee: Viacheslav Avramenko
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-24256.01.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> There is unquoted column name NWI_TABLE in one of the SQL queries which
> executed during REPL LOAD.
> This causes the command to fail when Postgres is used for metastore.
> {code:sql}
> SELECT \"NWI_NEXT\" FROM \"NEXT_WRITE_ID\" WHERE \"NWI_DATABASE\" = ? AND
> NWI_TABLE = ?
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-24256) REPL LOAD fails because of unquoted column name
[
https://issues.apache.org/jira/browse/HIVE-24256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17218327#comment-17218327
]
Viacheslav Avramenko commented on HIVE-24256:
-
Thanks, [~pvary] and [Miklos
Gergely!|https://reviews.apache.org/users/miki.gergely/]
> REPL LOAD fails because of unquoted column name
> ---
>
> Key: HIVE-24256
> URL: https://issues.apache.org/jira/browse/HIVE-24256
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Viacheslav Avramenko
>Assignee: Viacheslav Avramenko
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-24256.01.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> There is unquoted column name NWI_TABLE in one of the SQL queries which
> executed during REPL LOAD.
> This causes the command to fail when Postgres is used for metastore.
> {code:sql}
> SELECT \"NWI_NEXT\" FROM \"NEXT_WRITE_ID\" WHERE \"NWI_DATABASE\" = ? AND
> NWI_TABLE = ?
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-24256) REPL LOAD fails because of unquoted column name
[
https://issues.apache.org/jira/browse/HIVE-24256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214912#comment-17214912
]
Viacheslav Avramenko commented on HIVE-24256:
-
Having approval on review board [https://reviews.apache.org/r/72963/] could you
please let me know how to proceed from here?
> REPL LOAD fails because of unquoted column name
> ---
>
> Key: HIVE-24256
> URL: https://issues.apache.org/jira/browse/HIVE-24256
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Viacheslav Avramenko
>Assignee: Viacheslav Avramenko
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-24256.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There is unquoted column name NWI_TABLE in one of the SQL queries which
> executed during REPL LOAD.
> This causes the command to fail when Postgres is used for metastore.
> {code:sql}
> SELECT \"NWI_NEXT\" FROM \"NEXT_WRITE_ID\" WHERE \"NWI_DATABASE\" = ? AND
> NWI_TABLE = ?
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24256) REPL LOAD fails because of unquoted column name
[
https://issues.apache.org/jira/browse/HIVE-24256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viacheslav Avramenko updated HIVE-24256:
Attachment: HIVE-24256.01.patch
Fix Version/s: 4.0.0
Status: Patch Available (was: Open)
> REPL LOAD fails because of unquoted column name
> ---
>
> Key: HIVE-24256
> URL: https://issues.apache.org/jira/browse/HIVE-24256
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Viacheslav Avramenko
>Assignee: Viacheslav Avramenko
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-24256.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There is unquoted column name NWI_TABLE in one of the SQL queries which
> executed during REPL LOAD.
> This causes the command to fail when Postgres is used for metastore.
> {code:sql}
> SELECT \"NWI_NEXT\" FROM \"NEXT_WRITE_ID\" WHERE \"NWI_DATABASE\" = ? AND
> NWI_TABLE = ?
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HIVE-24256) REPL LOAD fails because of unquoted column name
[
https://issues.apache.org/jira/browse/HIVE-24256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-24256 started by Viacheslav Avramenko.
---
> REPL LOAD fails because of unquoted column name
> ---
>
> Key: HIVE-24256
> URL: https://issues.apache.org/jira/browse/HIVE-24256
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Viacheslav Avramenko
>Assignee: Viacheslav Avramenko
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There is unquoted column name NWI_TABLE in one of the SQL queries which
> executed during REPL LOAD.
> This causes the command to fail when Postgres is used for metastore.
> {code:sql}
> SELECT \"NWI_NEXT\" FROM \"NEXT_WRITE_ID\" WHERE \"NWI_DATABASE\" = ? AND
> NWI_TABLE = ?
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work stopped] (HIVE-24256) REPL LOAD fails because of unquoted column name
[
https://issues.apache.org/jira/browse/HIVE-24256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-24256 stopped by Viacheslav Avramenko.
---
> REPL LOAD fails because of unquoted column name
> ---
>
> Key: HIVE-24256
> URL: https://issues.apache.org/jira/browse/HIVE-24256
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Viacheslav Avramenko
>Assignee: Viacheslav Avramenko
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There is unquoted column name NWI_TABLE in one of the SQL queries which
> executed during REPL LOAD.
> This causes the command to fail when Postgres is used for metastore.
> {code:sql}
> SELECT \"NWI_NEXT\" FROM \"NEXT_WRITE_ID\" WHERE \"NWI_DATABASE\" = ? AND
> NWI_TABLE = ?
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24256) REPL LOAD fails because of unquoted column name
[
https://issues.apache.org/jira/browse/HIVE-24256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viacheslav Avramenko updated HIVE-24256:
Affects Version/s: 4.0.0
> REPL LOAD fails because of unquoted column name
> ---
>
> Key: HIVE-24256
> URL: https://issues.apache.org/jira/browse/HIVE-24256
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Viacheslav Avramenko
>Assignee: Viacheslav Avramenko
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There is unquoted column name NWI_TABLE in one of the SQL queries which
> executed during REPL LOAD.
> This causes the command to fail when Postgres is used for metastore.
> {code:sql}
> SELECT \"NWI_NEXT\" FROM \"NEXT_WRITE_ID\" WHERE \"NWI_DATABASE\" = ? AND
> NWI_TABLE = ?
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24256) REPL LOAD fails because of unquoted column name
[
https://issues.apache.org/jira/browse/HIVE-24256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viacheslav Avramenko updated HIVE-24256:
Description:
There is unquoted column name NWI_TABLE in one of the SQL queries which
executed during REPL LOAD.
This causes the command to fail when Postgres is used for metastore.
{code:sql}
SELECT \"NWI_NEXT\" FROM \"NEXT_WRITE_ID\" WHERE \"NWI_DATABASE\" = ? AND
NWI_TABLE = ?
{code}
> REPL LOAD fails because of unquoted column name
> ---
>
> Key: HIVE-24256
> URL: https://issues.apache.org/jira/browse/HIVE-24256
> Project: Hive
> Issue Type: Bug
>Reporter: Viacheslav Avramenko
>Assignee: Viacheslav Avramenko
>Priority: Major
> Labels: pull-request-available
>
> There is unquoted column name NWI_TABLE in one of the SQL queries which
> executed during REPL LOAD.
> This causes the command to fail when Postgres is used for metastore.
> {code:sql}
> SELECT \"NWI_NEXT\" FROM \"NEXT_WRITE_ID\" WHERE \"NWI_DATABASE\" = ? AND
> NWI_TABLE = ?
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24256) REPL LOAD fails because of unquoted column name
[
https://issues.apache.org/jira/browse/HIVE-24256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viacheslav Avramenko updated HIVE-24256:
Description: (was:
{code:java}
CREATE EXTERNAL TABLE dbs(db_id bigint, db_location_uri string, name string,
owner_name string, owner_type string) STORED BY
'org.apache.hive.storage.jdbc.JdbcStorageHandler' TBLPROPERTIES
('hive.sql.database.type'='METASTORE', 'hive.sql.query'='SELECT `DB_ID`,
`DB_LOCATION_URI`, `NAME`, `OWNER_NAME`, `OWNER_TYPE` FROM `DBS`');
{code}
==> Wrong Result <==
{code:java}
set hive.limit.optimize.enable=true;
select * from dbs limit 1;
--
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--
Map 1 .. container SUCCEEDED 0 0 0 0 0 0
--
VERTICES: 01/01 [==>>] 100% ELAPSED TIME: 0.91 s
--
++--+---+-+-+
| dbs.db_id | dbs.db_location_uri | dbs.name | dbs.owner_name | dbs.owner_type |
++--+---+-+-+
++--+---+-+-+
{code}
==> Correct Result <==
{code:java}
set hive.limit.optimize.enable=false;
select * from dbs limit 1;
--
VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
--
Map 1 .. container SUCCEEDED 1 100
0 0
--
VERTICES: 01/01 [==>>] 100% ELAPSED TIME: 4.11 s
--+++---+-+-+
| dbs.db_id |dbs.db_location_uri | dbs.name |
dbs.owner_name | dbs.owner_type |
+++---+-+-+
| 1 | hdfs://abcd:8020/warehouse/tablespace/managed/hive | default |
public | ROLE|
+++---+-+-+{code})
> REPL LOAD fails because of unquoted column name
> ---
>
> Key: HIVE-24256
> URL: https://issues.apache.org/jira/browse/HIVE-24256
> Project: Hive
> Issue Type: Bug
>Reporter: Viacheslav Avramenko
>Assignee: Naresh P R
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-24256) REPL LOAD fails because of unquoted column name
[ https://issues.apache.org/jira/browse/HIVE-24256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viacheslav Avramenko reassigned HIVE-24256: --- Assignee: Viacheslav Avramenko (was: Naresh P R) > REPL LOAD fails because of unquoted column name > --- > > Key: HIVE-24256 > URL: https://issues.apache.org/jira/browse/HIVE-24256 > Project: Hive > Issue Type: Bug >Reporter: Viacheslav Avramenko >Assignee: Viacheslav Avramenko >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-24256) REPL LOAD fails because of unquoted column name
[
https://issues.apache.org/jira/browse/HIVE-24256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viacheslav Avramenko reassigned HIVE-24256:
---
> REPL LOAD fails because of unquoted column name
> ---
>
> Key: HIVE-24256
> URL: https://issues.apache.org/jira/browse/HIVE-24256
> Project: Hive
> Issue Type: Bug
>Reporter: Viacheslav Avramenko
>Assignee: Naresh P R
>Priority: Major
> Labels: pull-request-available
>
>
> {code:java}
> CREATE EXTERNAL TABLE dbs(db_id bigint, db_location_uri string, name string,
> owner_name string, owner_type string) STORED BY
> 'org.apache.hive.storage.jdbc.JdbcStorageHandler' TBLPROPERTIES
> ('hive.sql.database.type'='METASTORE', 'hive.sql.query'='SELECT `DB_ID`,
> `DB_LOCATION_URI`, `NAME`, `OWNER_NAME`, `OWNER_TYPE` FROM `DBS`');
> {code}
> ==> Wrong Result <==
> {code:java}
> set hive.limit.optimize.enable=true;
> select * from dbs limit 1;
> --
> VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
> --
> Map 1 .. container SUCCEEDED 0 0 0 0 0 0
> --
> VERTICES: 01/01 [==>>] 100% ELAPSED TIME: 0.91 s
> --
> ++--+---+-+-+
> | dbs.db_id | dbs.db_location_uri | dbs.name | dbs.owner_name |
> dbs.owner_type |
> ++--+---+-+-+
> ++--+---+-+-+
> {code}
> ==> Correct Result <==
> {code:java}
> set hive.limit.optimize.enable=false;
> select * from dbs limit 1;
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. container SUCCEEDED 1 100
> 0 0
> --
> VERTICES: 01/01 [==>>] 100% ELAPSED TIME: 4.11 s
> --+++---+-+-+
> | dbs.db_id |dbs.db_location_uri | dbs.name
> | dbs.owner_name | dbs.owner_type |
> +++---+-+-+
> | 1 | hdfs://abcd:8020/warehouse/tablespace/managed/hive | default
> | public | ROLE|
> +++---+-+-+{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
