[jira] [Updated] (HIVE-27825) Better error message for an empty quoted identifier
[
https://issues.apache.org/jira/browse/HIVE-27825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
okumin updated HIVE-27825:
--
Status: Patch Available (was: Open)
> Better error message for an empty quoted identifier
> ---
>
> Key: HIVE-27825
> URL: https://issues.apache.org/jira/browse/HIVE-27825
> Project: Hive
> Issue Type: Improvement
> Components: Parser
>Affects Versions: 4.0.0-beta-1
>Reporter: okumin
>Assignee: okumin
>Priority: Minor
> Labels: pull-request-available
>
> SemanticAnalyzer throws a StringIndexOutOfBoundsException when a quoted
> identifier is empty. It is likely to happen when a template system fails to
> fill a database name or table name. We should return a better message so that
> a user can easily identify where is wrong.
>
> {code:java}
> 0: jdbc:hive2://hive-hiveserver2:1/defaul> SELECT * FROM ``;
> Error: Error while compiling statement: FAILED:
> StringIndexOutOfBoundsException String index out of range: 0
> (state=42000,code=4) {code}
> This is the stacktrace on HiveServer2.
> {code:java}
> org.apache.hive.service.cli.HiveSQLException: Error while compiling
> statement: FAILED: StringIndexOutOfBoundsException String index out of range: > 0
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:376)
> ~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:214)
> ~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> ...
> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of
> range: 0
> at java.lang.String.charAt(String.java:658) ~[?:1.8.0_275]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.unescapeIdentifier(BaseSemanticAnalyzer.java:608)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.getDbTableNamePair(BaseSemanticAnalyzer.java:424)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]{code}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-9260) Implement the bloom filter for the ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17779740#comment-17779740 ] zhangbutao commented on HIVE-9260: -- [~Ferd] Will you continue to finish the ticket? I think bloom filter is more usefull to accelerate the parquet table query. > Implement the bloom filter for the ParquetSerde > --- > > Key: HIVE-9260 > URL: https://issues.apache.org/jira/browse/HIVE-9260 > Project: Hive > Issue Type: Sub-task > Components: Serializers/Deserializers >Reporter: Ferdinand Xu >Assignee: Ferdinand Xu >Priority: Major > Attachments: HIVE-9260.patch > > > Implement the boom filter for parquet -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27189) Remove duplicate debug log in Hive.isSubDIr
[
https://issues.apache.org/jira/browse/HIVE-27189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shuyouZZ resolved HIVE-27189.
-
Fix Version/s: All Versions
Assignee: shuyouZZ
Resolution: Fixed
> Remove duplicate debug log in Hive.isSubDIr
> ---
>
> Key: HIVE-27189
> URL: https://issues.apache.org/jira/browse/HIVE-27189
> Project: Hive
> Issue Type: Improvement
>Reporter: shuyouZZ
>Assignee: shuyouZZ
>Priority: Major
> Labels: pull-request-available
> Fix For: All Versions
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> In class {{{}org.apache.hadoop.hive.ql.metadata.HIve{}}}, invoke method
> {{isSubDir}} will print twice
> {code:java}
> LOG.debug("The source path is " + fullF1 + " and the destination path is " +
> fullF2);{code}
> we should remove the duplicate debug log.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27826) Upgrade to Parquet 1.13.1
[ https://issues.apache.org/jira/browse/HIVE-27826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao updated HIVE-27826: -- Attachment: mvn_dependency_tree.text > Upgrade to Parquet 1.13.1 > - > > Key: HIVE-27826 > URL: https://issues.apache.org/jira/browse/HIVE-27826 > Project: Hive > Issue Type: Improvement > Components: Parquet >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > Attachments: mvn_dependency_tree.text > > > Upgrade parquet to 1.13.1. Apache Iceberg also use this latest parquet > version. > [https://github.com/apache/iceberg/pull/7301] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27826) Upgrade to Parquet 1.13.1
[ https://issues.apache.org/jira/browse/HIVE-27826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27826: -- Labels: pull-request-available (was: ) > Upgrade to Parquet 1.13.1 > - > > Key: HIVE-27826 > URL: https://issues.apache.org/jira/browse/HIVE-27826 > Project: Hive > Issue Type: Improvement > Components: Parquet >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > > Upgrade parquet to 1.13.1. Apache Iceberg also use this latest parquet > version. > [https://github.com/apache/iceberg/pull/7301] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27826) Upgrade to Parquet 1.13.1
[ https://issues.apache.org/jira/browse/HIVE-27826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao reassigned HIVE-27826: - Assignee: zhangbutao > Upgrade to Parquet 1.13.1 > - > > Key: HIVE-27826 > URL: https://issues.apache.org/jira/browse/HIVE-27826 > Project: Hive > Issue Type: Improvement > Components: Parquet >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > > Upgrade parquet to 1.13.1. Apache Iceberg also use this latest parquet > version. > [https://github.com/apache/iceberg/pull/7301] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27826) Upgrade to Parquet 1.13.1
[ https://issues.apache.org/jira/browse/HIVE-27826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao updated HIVE-27826: -- Description: Upgrade parquet to 1.13.1. Apache Iceberg also use this latest parquet version. [https://github.com/apache/iceberg/pull/7301] was:Upgrade parquet to 1.13.1. Apache Iceberg also use this parquet version. > Upgrade to Parquet 1.13.1 > - > > Key: HIVE-27826 > URL: https://issues.apache.org/jira/browse/HIVE-27826 > Project: Hive > Issue Type: Improvement > Components: Parquet >Reporter: zhangbutao >Priority: Major > > Upgrade parquet to 1.13.1. Apache Iceberg also use this latest parquet > version. > [https://github.com/apache/iceberg/pull/7301] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27826) Upgrade to Parquet 1.13.1
zhangbutao created HIVE-27826: - Summary: Upgrade to Parquet 1.13.1 Key: HIVE-27826 URL: https://issues.apache.org/jira/browse/HIVE-27826 Project: Hive Issue Type: Improvement Components: Parquet Reporter: zhangbutao Upgrade parquet to 1.13.1. Apache Iceberg also use this parquet version. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27825) Better error message for an empty quoted identifier
[
https://issues.apache.org/jira/browse/HIVE-27825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27825:
--
Labels: pull-request-available (was: )
> Better error message for an empty quoted identifier
> ---
>
> Key: HIVE-27825
> URL: https://issues.apache.org/jira/browse/HIVE-27825
> Project: Hive
> Issue Type: Improvement
> Components: Parser
>Affects Versions: 4.0.0-beta-1
>Reporter: okumin
>Assignee: okumin
>Priority: Minor
> Labels: pull-request-available
>
> SemanticAnalyzer throws a StringIndexOutOfBoundsException when a quoted
> identifier is empty. It is likely to happen when a template system fails to
> fill a database name or table name. We should return a better message so that
> a user can easily identify where is wrong.
>
> {code:java}
> 0: jdbc:hive2://hive-hiveserver2:1/defaul> SELECT * FROM ``;
> Error: Error while compiling statement: FAILED:
> StringIndexOutOfBoundsException String index out of range: 0
> (state=42000,code=4) {code}
> This is the stacktrace on HiveServer2.
> {code:java}
> org.apache.hive.service.cli.HiveSQLException: Error while compiling
> statement: FAILED: StringIndexOutOfBoundsException String index out of range: > 0
> at
> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:376)
> ~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:214)
> ~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> ...
> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of
> range: 0
> at java.lang.String.charAt(String.java:658) ~[?:1.8.0_275]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.unescapeIdentifier(BaseSemanticAnalyzer.java:608)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.getDbTableNamePair(BaseSemanticAnalyzer.java:424)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]{code}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-27825) Better error message for an empty quoted identifier
okumin created HIVE-27825:
-
Summary: Better error message for an empty quoted identifier
Key: HIVE-27825
URL: https://issues.apache.org/jira/browse/HIVE-27825
Project: Hive
Issue Type: Improvement
Components: Parser
Affects Versions: 4.0.0-beta-1
Reporter: okumin
Assignee: okumin
SemanticAnalyzer throws a StringIndexOutOfBoundsException when a quoted
identifier is empty. It is likely to happen when a template system fails to
fill a database name or table name. We should return a better message so that a
user can easily identify where is wrong.
{code:java}
0: jdbc:hive2://hive-hiveserver2:1/defaul> SELECT * FROM ``;
Error: Error while compiling statement: FAILED: StringIndexOutOfBoundsException
String index out of range: 0 (state=42000,code=4) {code}
This is the stacktrace on HiveServer2.
{code:java}
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement:
FAILED: StringIndexOutOfBoundsException String index out of range: 0
at
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:376)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:214)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
...
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of
range: 0
at java.lang.String.charAt(String.java:658) ~[?:1.8.0_275]
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.unescapeIdentifier(BaseSemanticAnalyzer.java:608)
~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.getDbTableNamePair(BaseSemanticAnalyzer.java:424)
~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27824) Upgrade Ivy to 2.5.2
[ https://issues.apache.org/jira/browse/HIVE-27824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27824: -- Labels: pull-request-available (was: ) > Upgrade Ivy to 2.5.2 > > > Key: HIVE-27824 > URL: https://issues.apache.org/jira/browse/HIVE-27824 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Devaspati Krishnatri >Assignee: Devaspati Krishnatri >Priority: Major > Labels: pull-request-available > Attachments: mvn_dependency_tree.txt > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27824) Upgrade Ivy to 2.5.2
[ https://issues.apache.org/jira/browse/HIVE-27824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaspati Krishnatri updated HIVE-27824: Attachment: mvn_dependency_tree.txt > Upgrade Ivy to 2.5.2 > > > Key: HIVE-27824 > URL: https://issues.apache.org/jira/browse/HIVE-27824 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Devaspati Krishnatri >Assignee: Devaspati Krishnatri >Priority: Major > Attachments: mvn_dependency_tree.txt > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27824) Upgrade Ivy to 2.5.2
Devaspati Krishnatri created HIVE-27824: --- Summary: Upgrade Ivy to 2.5.2 Key: HIVE-27824 URL: https://issues.apache.org/jira/browse/HIVE-27824 Project: Hive Issue Type: Task Components: Hive Reporter: Devaspati Krishnatri Assignee: Devaspati Krishnatri -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27823) select returns struct unfilled fields with nulls, some way to suppress this?
Alexander Petrossian (PAF) created HIVE-27823:
-
Summary: select returns struct unfilled fields with nulls, some
way to suppress this?
Key: HIVE-27823
URL: https://issues.apache.org/jira/browse/HIVE-27823
Project: Hive
Issue Type: Wish
Affects Versions: 3.1.0
Reporter: Alexander Petrossian (PAF)
One of fields has struct type
{code:sql}
struct<
BoolValue:struct,
IntValue:struct,
Ipv4Value:struct,
StringValue:struct,
ZonedTimeValue:struct>
>
{code}
Of which only one at a time is used.
As result of select we get all fields, others filled with nulls:
{code:json}
{
"BoolValue": null,
"IntValue": null,
"Ipv4Value": null,
"StringValue": {
"value": "ipv4:10.31.94.28:31003"
},
"ZonedTimeValue": null
},
{code}
This is truth, but an unreadable one.
Any ideas on how to get this output for this field:
{code:json}
{
"StringValue": {
"value": "ipv4:10.31.94.28:31003"
}
},
{code}
??
Thanks!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-22336) Updates should be pushed to the Metastore backend DB before creating the notification event
[ https://issues.apache.org/jira/browse/HIVE-22336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22336: -- Labels: pull-request-available (was: ) > Updates should be pushed to the Metastore backend DB before creating the > notification event > --- > > Key: HIVE-22336 > URL: https://issues.apache.org/jira/browse/HIVE-22336 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Marta Kuczora >Assignee: Marta Kuczora >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > Attachments: HIVE-22336.1.patch, HIVE-22336.2.patch, > HIVE-22336.3.patch > > > There was an issue on HDP-3.1 where a table couldn't be deleted, because some > related objects (like storage descriptor) were missing from the metastore. > There was a previous delete attempt on that table which went wrong, but no > rollback happened, that's why the SD were missing. In that previous delete, > the notification creation swallowed the error which came from the backend DB, > that's why no rollback happened. Here are the steps which happened in the > first delete attempt: > > # Open a transaction (transaction_1) - this step was successful > # Delete all the objects which are related to the table - this step was > successful too, so the SD and other objects were deleted > # Delete the table - this step failed in the backend DB, but according to the > log the delete happens in a batch statement, so it won't necessarily be > executed right at this moment, so we won't see an error here > # Create a notification about the table delete: > ## Open an other transaction for the notification creation (transaction_2) - > call the ObjectStore.openTransaction method which increases a counter for > open transactions and then checks if there is already an active transaction. > If there is, then just returns true and doesn't really create a new > transaction. > ## Lock the notification id in the metastore backend db for update - here is > where the exception from the backend DB (let's call it "MySQL Exception") > manifests > ## If an exception occurs during acquiring the log, retry - The "MySQL > Exception" was caught and since there is no check on the exception, the retry > mechanism thinks that it happened because couldn't acquire the log for the > notification id, so retries and "forgot" about the "MySQL Exception". > ## If the lock was acquired successfully, create the notification - Second > time, the lock was acquired successfully, so the notification creation was > successful. > ## Commit transaction_2 - Just decrease the transaction counter, but doesn't > actually commits anything. > # Commit transaction_1 - This commits the transaction, but since the error > already got manifested and kind of "handled", here we won't see any error, > just that the commit was successful, so no rollback happens and leaves the > table object in an invalid state. > # If the commit was not successful then rollback > In the customer setup, this issue could be fixed by adding a flush call > before creating the notification event, so all the updates would be pushed to > the backend db and the error would manifest at this point. With this, the > error would go back to the HiveMetastore class which would do the rollback > and the delete table operation would fail as it should be, since the table > couldn't be deleted. But then the Hivemetastore retry mechanism could try the > table deletion again. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27822) Backport of HIVE-22336: Updates should be pushed to the Metastore backend DB before creating the notification event
Diksha created HIVE-27822: - Summary: Backport of HIVE-22336: Updates should be pushed to the Metastore backend DB before creating the notification event Key: HIVE-27822 URL: https://issues.apache.org/jira/browse/HIVE-27822 Project: Hive Issue Type: Sub-task Reporter: Diksha Assignee: Diksha Fix For: 3.2.0 This is a backport of HIVE-22336: Updates should be pushed to the Metastore backend DB before creating the notification event -- This message was sent by Atlassian Jira (v8.20.10#820010)
