[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=778052&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-778052
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 03/Jun/22 12:00
Start Date: 03/Jun/22 12:00
Worklog Time Spent: 10m
Work Description: deniskuzZ merged PR #3325:
URL: https://github.com/apache/hive/pull/3325
Issue Time Tracking
---
Worklog Id: (was: 778052)
Time Spent: 2h 40m (was: 2.5h)
> Addendum to HIVE-26107: perpared statement is not working on Postgres
> -
>
> Key: HIVE-26267
> URL: https://issues.apache.org/jira/browse/HIVE-26267
> Project: Hive
> Issue Type: Bug
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> The assembled prepared statement in
> {code:java}
> org.apache.hadoop.hive.metastore.txn.TxnHandler#compact{code}
> does not work for Postgres DB.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777958&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777958
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 03/Jun/22 07:54
Start Date: 03/Jun/22 07:54
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r888707129
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3701,122 +3698,127 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
* compactions for any resource.
*/
handle =
getMutexAPI().acquireLock(MUTEX_KEY.CompactionScheduler.name());
-dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
-stmt = dbConn.createStatement();
-long id = generateCompactionQueueId(stmt);
-
-GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
-Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
-final ValidCompactorWriteIdList tblValidWriteIds =
-
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
-LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
+try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
+ try (Statement stmt = dbConn.createStatement()) {
+
+long id = generateCompactionQueueId(stmt);
+
+GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
+Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
+final ValidCompactorWriteIdList tblValidWriteIds =
+
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
+LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
+
+StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\",
\"CQ_STATE\" FROM \"COMPACTION_QUEUE\" WHERE").
+append(" (\"CQ_STATE\" IN(").
+
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
+append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
+append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
+append(" AND \"CQ_DATABASE\"=?").
+append(" AND \"CQ_TABLE\"=?").append(" AND ");
+if(rqst.getPartitionname() == null) {
+ sb.append("\"CQ_PARTITION\" is null");
+} else {
+ sb.append("\"CQ_PARTITION\"=?");
+}
-List params = new ArrayList<>();
-StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
- append(" (\"CQ_STATE\" IN(").
-
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
-append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
-append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
-append(" AND \"CQ_DATABASE\"=?").
-append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
-if(rqst.getPartitionname() == null) {
- sb.append("\"CQ_PARTITION\" is null");
-} else {
- sb.append("\"CQ_PARTITION\"=?");
- params.add(rqst.getPartitionname());
-}
+try (PreparedStatement pst =
dbConn.prepareStatement(sqlGenerator.addEscapeCharacters(sb.toString( {
+ pst.setLong(1, tblValidWriteIds.getHighWatermark());
+ pst.setString(2, rqst.getDbname());
+ pst.setString(3, rqst.getTablename());
+ if (rqst.getPartitionname() != null) {
+pst.setString(4, rqst.getPartitionname());
+ }
+ LOG.debug("Going to execute query <" + sb + ">");
+ try (ResultSet rs = pst.executeQuery()) {
+if(rs.next()) {
+ long enqueuedId = rs.getLong(1);
+ String state =
compactorStateToResponse(rs.getString(2).charAt(0));
+ LOG.info("Ignoring request to compact " + rqst.getDbname() +
"/" + rqst.getTablename() +
+ "/" + rqst.getPartitionname() + " since it is already "
+ quoteString(state) +
+ " with id=" + enqueuedId);
+ CompactionResponse resp = new CompactionResponse(-1,
REFUSED_RESPONSE, false);
+ resp.setErrormessage("Compaction is already scheduled with
state=" + quoteString(state) +
+
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777955&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777955
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 03/Jun/22 07:45
Start Date: 03/Jun/22 07:45
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r888700909
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3701,122 +3698,127 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
* compactions for any resource.
*/
handle =
getMutexAPI().acquireLock(MUTEX_KEY.CompactionScheduler.name());
-dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
-stmt = dbConn.createStatement();
-long id = generateCompactionQueueId(stmt);
-
-GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
-Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
-final ValidCompactorWriteIdList tblValidWriteIds =
-
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
-LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
+try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
+ try (Statement stmt = dbConn.createStatement()) {
+
+long id = generateCompactionQueueId(stmt);
+
+GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
+Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
+final ValidCompactorWriteIdList tblValidWriteIds =
+
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
+LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
+
+StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\",
\"CQ_STATE\" FROM \"COMPACTION_QUEUE\" WHERE").
+append(" (\"CQ_STATE\" IN(").
+
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
+append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
+append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
+append(" AND \"CQ_DATABASE\"=?").
+append(" AND \"CQ_TABLE\"=?").append(" AND ");
+if(rqst.getPartitionname() == null) {
+ sb.append("\"CQ_PARTITION\" is null");
+} else {
+ sb.append("\"CQ_PARTITION\"=?");
+}
-List params = new ArrayList<>();
-StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
- append(" (\"CQ_STATE\" IN(").
-
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
-append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
-append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
-append(" AND \"CQ_DATABASE\"=?").
-append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
-if(rqst.getPartitionname() == null) {
- sb.append("\"CQ_PARTITION\" is null");
-} else {
- sb.append("\"CQ_PARTITION\"=?");
- params.add(rqst.getPartitionname());
-}
+try (PreparedStatement pst =
dbConn.prepareStatement(sqlGenerator.addEscapeCharacters(sb.toString( {
+ pst.setLong(1, tblValidWriteIds.getHighWatermark());
+ pst.setString(2, rqst.getDbname());
+ pst.setString(3, rqst.getTablename());
+ if (rqst.getPartitionname() != null) {
+pst.setString(4, rqst.getPartitionname());
+ }
+ LOG.debug("Going to execute query <" + sb + ">");
+ try (ResultSet rs = pst.executeQuery()) {
+if(rs.next()) {
+ long enqueuedId = rs.getLong(1);
+ String state =
compactorStateToResponse(rs.getString(2).charAt(0));
+ LOG.info("Ignoring request to compact " + rqst.getDbname() +
"/" + rqst.getTablename() +
+ "/" + rqst.getPartitionname() + " since it is already "
+ quoteString(state) +
+ " with id=" + enqueuedId);
+ CompactionResponse resp = new CompactionResponse(-1,
REFUSED_RESPONSE, false);
+ resp.setErrormessage("Compaction is already scheduled with
state=" + quoteString(state) +
+
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777956&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777956
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 03/Jun/22 07:45
Start Date: 03/Jun/22 07:45
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r888700909
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3701,122 +3698,127 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
* compactions for any resource.
*/
handle =
getMutexAPI().acquireLock(MUTEX_KEY.CompactionScheduler.name());
-dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
-stmt = dbConn.createStatement();
-long id = generateCompactionQueueId(stmt);
-
-GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
-Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
-final ValidCompactorWriteIdList tblValidWriteIds =
-
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
-LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
+try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
+ try (Statement stmt = dbConn.createStatement()) {
+
+long id = generateCompactionQueueId(stmt);
+
+GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
+Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
+final ValidCompactorWriteIdList tblValidWriteIds =
+
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
+LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
+
+StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\",
\"CQ_STATE\" FROM \"COMPACTION_QUEUE\" WHERE").
+append(" (\"CQ_STATE\" IN(").
+
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
+append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
+append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
+append(" AND \"CQ_DATABASE\"=?").
+append(" AND \"CQ_TABLE\"=?").append(" AND ");
+if(rqst.getPartitionname() == null) {
+ sb.append("\"CQ_PARTITION\" is null");
+} else {
+ sb.append("\"CQ_PARTITION\"=?");
+}
-List params = new ArrayList<>();
-StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
- append(" (\"CQ_STATE\" IN(").
-
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
-append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
-append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
-append(" AND \"CQ_DATABASE\"=?").
-append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
-if(rqst.getPartitionname() == null) {
- sb.append("\"CQ_PARTITION\" is null");
-} else {
- sb.append("\"CQ_PARTITION\"=?");
- params.add(rqst.getPartitionname());
-}
+try (PreparedStatement pst =
dbConn.prepareStatement(sqlGenerator.addEscapeCharacters(sb.toString( {
+ pst.setLong(1, tblValidWriteIds.getHighWatermark());
+ pst.setString(2, rqst.getDbname());
+ pst.setString(3, rqst.getTablename());
+ if (rqst.getPartitionname() != null) {
+pst.setString(4, rqst.getPartitionname());
+ }
+ LOG.debug("Going to execute query <" + sb + ">");
+ try (ResultSet rs = pst.executeQuery()) {
+if(rs.next()) {
+ long enqueuedId = rs.getLong(1);
+ String state =
compactorStateToResponse(rs.getString(2).charAt(0));
+ LOG.info("Ignoring request to compact " + rqst.getDbname() +
"/" + rqst.getTablename() +
+ "/" + rqst.getPartitionname() + " since it is already "
+ quoteString(state) +
+ " with id=" + enqueuedId);
+ CompactionResponse resp = new CompactionResponse(-1,
REFUSED_RESPONSE, false);
+ resp.setErrormessage("Compaction is already scheduled with
state=" + quoteString(state) +
+
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777954&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777954
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 03/Jun/22 07:44
Start Date: 03/Jun/22 07:44
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r888700266
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3701,122 +3698,125 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
* compactions for any resource.
*/
handle =
getMutexAPI().acquireLock(MUTEX_KEY.CompactionScheduler.name());
-dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
-stmt = dbConn.createStatement();
-
-long id = generateCompactionQueueId(stmt);
-
-GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
-Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
-final ValidCompactorWriteIdList tblValidWriteIds =
-
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
-LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
-
-List params = new ArrayList<>();
-StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
- append(" (\"CQ_STATE\" IN(").
-
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
-append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
-append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
-append(" AND \"CQ_DATABASE\"=?").
-append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
-if(rqst.getPartitionname() == null) {
- sb.append("\"CQ_PARTITION\" is null");
-} else {
- sb.append("\"CQ_PARTITION\"=?");
- params.add(rqst.getPartitionname());
-}
+try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
Review Comment:
i've missed the. rollback part
Issue Time Tracking
---
Worklog Id: (was: 777954)
Time Spent: 2h (was: 1h 50m)
> Addendum to HIVE-26107: perpared statement is not working on Postgres
> -
>
> Key: HIVE-26267
> URL: https://issues.apache.org/jira/browse/HIVE-26267
> Project: Hive
> Issue Type: Bug
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> The assembled prepared statement in
> {code:java}
> org.apache.hadoop.hive.metastore.txn.TxnHandler#compact{code}
> does not work for Postgres DB.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777427&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777427
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 02/Jun/22 11:38
Start Date: 02/Jun/22 11:38
Worklog Time Spent: 10m
Work Description: veghlaci05 commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r887852145
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3701,122 +3698,125 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
* compactions for any resource.
*/
handle =
getMutexAPI().acquireLock(MUTEX_KEY.CompactionScheduler.name());
-dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
-stmt = dbConn.createStatement();
-
-long id = generateCompactionQueueId(stmt);
-
-GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
-Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
-final ValidCompactorWriteIdList tblValidWriteIds =
-
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
-LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
-
-List params = new ArrayList<>();
-StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
- append(" (\"CQ_STATE\" IN(").
-
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
-append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
-append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
-append(" AND \"CQ_DATABASE\"=?").
-append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
-if(rqst.getPartitionname() == null) {
- sb.append("\"CQ_PARTITION\" is null");
-} else {
- sb.append("\"CQ_PARTITION\"=?");
- params.add(rqst.getPartitionname());
-}
+try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
Review Comment:
I have googled for it before splitting, and this approach was the suggested
one. if you check the java
[spec](https://docs.oracle.com/javase/specs/jls/se7/html/jls-14.html#jls-14.20.3.2),
the extended try-with -resources construct is translated to a nested
try-catch. The inner try-catch is responsible for closing the resources, so
when the outer catch clause is executed (`dbConn.rollback();`), the connection
object should be already closed. So as I understand, you cannot execute code in
the catch/finally blocks which require the resource to be open. Did I miss
something here?
Issue Time Tracking
---
Worklog Id: (was: 777427)
Time Spent: 1h 50m (was: 1h 40m)
> Addendum to HIVE-26107: perpared statement is not working on Postgres
> -
>
> Key: HIVE-26267
> URL: https://issues.apache.org/jira/browse/HIVE-26267
> Project: Hive
> Issue Type: Bug
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The assembled prepared statement in
> {code:java}
> org.apache.hadoop.hive.metastore.txn.TxnHandler#compact{code}
> does not work for Postgres DB.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777395&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777395
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 02/Jun/22 10:43
Start Date: 02/Jun/22 10:43
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r887809222
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3701,122 +3698,125 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
* compactions for any resource.
*/
handle =
getMutexAPI().acquireLock(MUTEX_KEY.CompactionScheduler.name());
-dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
-stmt = dbConn.createStatement();
-
-long id = generateCompactionQueueId(stmt);
-
-GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
-Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
-final ValidCompactorWriteIdList tblValidWriteIds =
-
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
-LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
-
-List params = new ArrayList<>();
-StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
- append(" (\"CQ_STATE\" IN(").
-
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
-append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
-append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
-append(" AND \"CQ_DATABASE\"=?").
-append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
-if(rqst.getPartitionname() == null) {
- sb.append("\"CQ_PARTITION\" is null");
-} else {
- sb.append("\"CQ_PARTITION\"=?");
- params.add(rqst.getPartitionname());
-}
+try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
Review Comment:
no need for an extra try, you could initialize dbConn and stmt in a single
try:
try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED);
Statement stmt = dbConn.createStatement()) {
Issue Time Tracking
---
Worklog Id: (was: 777395)
Time Spent: 1h 40m (was: 1.5h)
> Addendum to HIVE-26107: perpared statement is not working on Postgres
> -
>
> Key: HIVE-26267
> URL: https://issues.apache.org/jira/browse/HIVE-26267
> Project: Hive
> Issue Type: Bug
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> The assembled prepared statement in
> {code:java}
> org.apache.hadoop.hive.metastore.txn.TxnHandler#compact{code}
> does not work for Postgres DB.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777393&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777393
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 02/Jun/22 10:43
Start Date: 02/Jun/22 10:43
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r887809222
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3701,122 +3698,125 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
* compactions for any resource.
*/
handle =
getMutexAPI().acquireLock(MUTEX_KEY.CompactionScheduler.name());
-dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
-stmt = dbConn.createStatement();
-
-long id = generateCompactionQueueId(stmt);
-
-GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
-Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
-final ValidCompactorWriteIdList tblValidWriteIds =
-
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
-LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
-
-List params = new ArrayList<>();
-StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
- append(" (\"CQ_STATE\" IN(").
-
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
-append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
-append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
-append(" AND \"CQ_DATABASE\"=?").
-append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
-if(rqst.getPartitionname() == null) {
- sb.append("\"CQ_PARTITION\" is null");
-} else {
- sb.append("\"CQ_PARTITION\"=?");
- params.add(rqst.getPartitionname());
-}
+try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
Review Comment:
no need for an extra try, you could initialize dbConn and stmt in single try:
try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED);
Statement stmt = dbConn.createStatement()) {
Issue Time Tracking
---
Worklog Id: (was: 777393)
Time Spent: 1.5h (was: 1h 20m)
> Addendum to HIVE-26107: perpared statement is not working on Postgres
> -
>
> Key: HIVE-26267
> URL: https://issues.apache.org/jira/browse/HIVE-26267
> Project: Hive
> Issue Type: Bug
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The assembled prepared statement in
> {code:java}
> org.apache.hadoop.hive.metastore.txn.TxnHandler#compact{code}
> does not work for Postgres DB.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777388&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777388
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 02/Jun/22 10:36
Start Date: 02/Jun/22 10:36
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r887809222
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3701,122 +3698,125 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
* compactions for any resource.
*/
handle =
getMutexAPI().acquireLock(MUTEX_KEY.CompactionScheduler.name());
-dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
-stmt = dbConn.createStatement();
-
-long id = generateCompactionQueueId(stmt);
-
-GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
-Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
-final ValidCompactorWriteIdList tblValidWriteIds =
-
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
-LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
-
-List params = new ArrayList<>();
-StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
- append(" (\"CQ_STATE\" IN(").
-
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
-append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
-append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
-append(" AND \"CQ_DATABASE\"=?").
-append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
-if(rqst.getPartitionname() == null) {
- sb.append("\"CQ_PARTITION\" is null");
-} else {
- sb.append("\"CQ_PARTITION\"=?");
- params.add(rqst.getPartitionname());
-}
+try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
Review Comment:
no need for an extra try, you could initialize dbConn and stmt in the above
block (line 3693) in single try:
try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED);
Statement stmt = dbConn.createStatement()) {
Issue Time Tracking
---
Worklog Id: (was: 777388)
Time Spent: 1h 20m (was: 1h 10m)
> Addendum to HIVE-26107: perpared statement is not working on Postgres
> -
>
> Key: HIVE-26267
> URL: https://issues.apache.org/jira/browse/HIVE-26267
> Project: Hive
> Issue Type: Bug
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The assembled prepared statement in
> {code:java}
> org.apache.hadoop.hive.metastore.txn.TxnHandler#compact{code}
> does not work for Postgres DB.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777387&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777387
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 02/Jun/22 10:34
Start Date: 02/Jun/22 10:34
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r887809222
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3701,122 +3698,125 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
* compactions for any resource.
*/
handle =
getMutexAPI().acquireLock(MUTEX_KEY.CompactionScheduler.name());
-dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
-stmt = dbConn.createStatement();
-
-long id = generateCompactionQueueId(stmt);
-
-GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
-Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
-final ValidCompactorWriteIdList tblValidWriteIds =
-
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
-LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
-
-List params = new ArrayList<>();
-StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
- append(" (\"CQ_STATE\" IN(").
-
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
-append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
-append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
-append(" AND \"CQ_DATABASE\"=?").
-append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
-if(rqst.getPartitionname() == null) {
- sb.append("\"CQ_PARTITION\" is null");
-} else {
- sb.append("\"CQ_PARTITION\"=?");
- params.add(rqst.getPartitionname());
-}
+try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
Review Comment:
no need for an extra try, you could initialize dbConn in the above block
Issue Time Tracking
---
Worklog Id: (was: 777387)
Time Spent: 1h 10m (was: 1h)
> Addendum to HIVE-26107: perpared statement is not working on Postgres
> -
>
> Key: HIVE-26267
> URL: https://issues.apache.org/jira/browse/HIVE-26267
> Project: Hive
> Issue Type: Bug
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The assembled prepared statement in
> {code:java}
> org.apache.hadoop.hive.metastore.txn.TxnHandler#compact{code}
> does not work for Postgres DB.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777386&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777386
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 02/Jun/22 10:28
Start Date: 02/Jun/22 10:28
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r887804677
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3701,122 +3698,125 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
* compactions for any resource.
*/
handle =
getMutexAPI().acquireLock(MUTEX_KEY.CompactionScheduler.name());
-dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
-stmt = dbConn.createStatement();
-
-long id = generateCompactionQueueId(stmt);
-
-GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
-Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
-final ValidCompactorWriteIdList tblValidWriteIds =
-
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
-LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
-
-List params = new ArrayList<>();
-StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
- append(" (\"CQ_STATE\" IN(").
-
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
-append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
-append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
-append(" AND \"CQ_DATABASE\"=?").
-append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
-if(rqst.getPartitionname() == null) {
- sb.append("\"CQ_PARTITION\" is null");
-} else {
- sb.append("\"CQ_PARTITION\"=?");
- params.add(rqst.getPartitionname());
-}
+try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
+ try (Statement stmt = dbConn.createStatement()) {
+
+long id = generateCompactionQueueId(stmt);
+
+GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
+Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
+final ValidCompactorWriteIdList tblValidWriteIds =
+
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
+LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
+
+StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\",
\"CQ_STATE\" FROM \"COMPACTION_QUEUE\" WHERE").
+append(" (\"CQ_STATE\" IN(").
+
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
+append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
+append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
+append(" AND \"CQ_DATABASE\"=?").
+append(" AND \"CQ_TABLE\"=?").append(" AND ");
+if(rqst.getPartitionname() == null) {
+ sb.append("\"CQ_PARTITION\" is null");
+} else {
+ sb.append("\"CQ_PARTITION\"=?");
+}
-pst = sqlGenerator.prepareStmtWithParameters(dbConn, sb.toString(),
params);
-LOG.debug("Going to execute query <" + sb + ">");
-ResultSet rs = pst.executeQuery();
-if(rs.next()) {
- long enqueuedId = rs.getLong(1);
- String state = compactorStateToResponse(rs.getString(2).charAt(0));
- LOG.info("Ignoring request to compact " + rqst.getDbname() + "/" +
rqst.getTablename() +
-"/" + rqst.getPartitionname() + " since it is already " +
quoteString(state) +
-" with id=" + enqueuedId);
- CompactionResponse resp = new CompactionResponse(-1,
REFUSED_RESPONSE, false);
- resp.setErrormessage("Compaction is already scheduled with state=" +
quoteString(state) +
- " and id=" + enqueuedId);
- return resp;
-}
-close(rs);
-closeStmt(pst);
-params.clear();
-StringBuilder buf = new StringBuilder("INSERT INTO
\"COMPACTION_QUEUE\" (\"CQ_ID\", \"CQ_DATABASE\", " +
- "\"CQ_TABLE\", ");
-String partName = rqst.getPartitionname();
-if (partName != null) buf.append("\"CQ_PARTITION\", ");
-buf.append("\"
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777385&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777385
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 02/Jun/22 10:27
Start Date: 02/Jun/22 10:27
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r887804398
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3701,122 +3698,125 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
* compactions for any resource.
*/
handle =
getMutexAPI().acquireLock(MUTEX_KEY.CompactionScheduler.name());
-dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
-stmt = dbConn.createStatement();
-
-long id = generateCompactionQueueId(stmt);
-
-GetValidWriteIdsRequest request = new GetValidWriteIdsRequest(
-Collections.singletonList(getFullTableName(rqst.getDbname(),
rqst.getTablename(;
-final ValidCompactorWriteIdList tblValidWriteIds =
-
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
-LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
-
-List params = new ArrayList<>();
-StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
- append(" (\"CQ_STATE\" IN(").
-
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
-append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
-append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
-append(" AND \"CQ_DATABASE\"=?").
-append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
-if(rqst.getPartitionname() == null) {
- sb.append("\"CQ_PARTITION\" is null");
-} else {
- sb.append("\"CQ_PARTITION\"=?");
- params.add(rqst.getPartitionname());
-}
+try (Connection dbConn =
getDbConn(Connection.TRANSACTION_READ_COMMITTED)) {
+ try (Statement stmt = dbConn.createStatement()) {
+
Review Comment:
nit: new line
Issue Time Tracking
---
Worklog Id: (was: 777385)
Time Spent: 50m (was: 40m)
> Addendum to HIVE-26107: perpared statement is not working on Postgres
> -
>
> Key: HIVE-26267
> URL: https://issues.apache.org/jira/browse/HIVE-26267
> Project: Hive
> Issue Type: Bug
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The assembled prepared statement in
> {code:java}
> org.apache.hadoop.hive.metastore.txn.TxnHandler#compact{code}
> does not work for Postgres DB.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777343&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777343
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 02/Jun/22 08:27
Start Date: 02/Jun/22 08:27
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r887702617
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3712,25 +3712,26 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
-List params = new ArrayList<>();
StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
append(" (\"CQ_STATE\" IN(").
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
append(" AND \"CQ_DATABASE\"=?").
append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
if(rqst.getPartitionname() == null) {
sb.append("\"CQ_PARTITION\" is null");
} else {
sb.append("\"CQ_PARTITION\"=?");
Review Comment:
no need to check explicitly for null, you can use `pstmt.setObject()`
Issue Time Tracking
---
Worklog Id: (was: 777343)
Time Spent: 40m (was: 0.5h)
> Addendum to HIVE-26107: perpared statement is not working on Postgres
> -
>
> Key: HIVE-26267
> URL: https://issues.apache.org/jira/browse/HIVE-26267
> Project: Hive
> Issue Type: Bug
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The assembled prepared statement in
> {code:java}
> org.apache.hadoop.hive.metastore.txn.TxnHandler#compact{code}
> does not work for Postgres DB.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777334&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777334
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 02/Jun/22 08:09
Start Date: 02/Jun/22 08:09
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r887686514
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3746,7 +3747,7 @@ public CompactionResponse compact(CompactionRequest rqst)
throws MetaException {
}
close(rs);
Review Comment:
could we wrap prep statement in try-catch so no explicit close is required
Issue Time Tracking
---
Worklog Id: (was: 777334)
Time Spent: 0.5h (was: 20m)
> Addendum to HIVE-26107: perpared statement is not working on Postgres
> -
>
> Key: HIVE-26267
> URL: https://issues.apache.org/jira/browse/HIVE-26267
> Project: Hive
> Issue Type: Bug
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The assembled prepared statement in
> {code:java}
> org.apache.hadoop.hive.metastore.txn.TxnHandler#compact{code}
> does not work for Postgres DB.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=777333&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777333
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 02/Jun/22 08:07
Start Date: 02/Jun/22 08:07
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3325:
URL: https://github.com/apache/hive/pull/3325#discussion_r887685084
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:
##
@@ -3712,25 +3712,26 @@ public CompactionResponse compact(CompactionRequest
rqst) throws MetaException {
TxnUtils.createValidCompactWriteIdList(getValidWriteIds(request).getTblValidWriteIds().get(0));
LOG.debug("ValidCompactWriteIdList: " +
tblValidWriteIds.writeToString());
-List params = new ArrayList<>();
StringBuilder sb = new StringBuilder("SELECT \"CQ_ID\", \"CQ_STATE\"
FROM \"COMPACTION_QUEUE\" WHERE").
append(" (\"CQ_STATE\" IN(").
append(quoteChar(INITIATED_STATE)).append(",").append(quoteChar(WORKING_STATE)).
append(") OR (\"CQ_STATE\" =
").append(quoteChar(READY_FOR_CLEANING)).
append(" AND \"CQ_HIGHEST_WRITE_ID\" = ?))").
append(" AND \"CQ_DATABASE\"=?").
append(" AND \"CQ_TABLE\"=?").append(" AND ");
-params.add(Long.toString(tblValidWriteIds.getHighWatermark()));
-params.add(rqst.getDbname());
-params.add(rqst.getTablename());
if(rqst.getPartitionname() == null) {
sb.append("\"CQ_PARTITION\" is null");
} else {
sb.append("\"CQ_PARTITION\"=?");
- params.add(rqst.getPartitionname());
}
-pst = sqlGenerator.prepareStmtWithParameters(dbConn, sb.toString(),
params);
+pst =
dbConn.prepareStatement(sqlGenerator.addEscapeCharacters(sb.toString()));
+pst.setLong(1, tblValidWriteIds.getHighWatermark());
+pst.setString(2, rqst.getDbname());
+pst.setString(3, rqst.getTablename());
+if(rqst.getPartitionname() != null) {
Review Comment:
nit: space
Issue Time Tracking
---
Worklog Id: (was: 777333)
Time Spent: 20m (was: 10m)
> Addendum to HIVE-26107: perpared statement is not working on Postgres
> -
>
> Key: HIVE-26267
> URL: https://issues.apache.org/jira/browse/HIVE-26267
> Project: Hive
> Issue Type: Bug
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The assembled prepared statement in
> {code:java}
> org.apache.hadoop.hive.metastore.txn.TxnHandler#compact{code}
> does not work for Postgres DB.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-26267) Addendum to HIVE-26107: perpared statement is not working on Postgres
[
https://issues.apache.org/jira/browse/HIVE-26267?focusedWorklogId=774949&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774949
]
ASF GitHub Bot logged work on HIVE-26267:
-
Author: ASF GitHub Bot
Created on: 26/May/22 08:51
Start Date: 26/May/22 08:51
Worklog Time Spent: 10m
Work Description: veghlaci05 opened a new pull request, #3325:
URL: https://github.com/apache/hive/pull/3325
### What changes were proposed in this pull request?
In org.apache.hadoop.hive.metastore.txn.TxnHandler#compact() use the
java.sql.PreparedStatement#setXXX methods instead of the
org.apache.hadoop.hive.metastore.tools.SQLGenerator#prepareStmtWithParameters()
method.
### Why are the changes needed?
The SQL statement modified by HIVE-26107 in
org.apache.hadoop.hive.metastore.txn.TxnHandler#compact is not working with
Postgres DB.
org.apache.hadoop.hive.metastore.tools.SQLGenerator#prepareStmtWithParameters()
can handle only string parameters.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Manually and through unit tests
Issue Time Tracking
---
Worklog Id: (was: 774949)
Remaining Estimate: 0h
Time Spent: 10m
> Addendum to HIVE-26107: perpared statement is not working on Postgres
> -
>
> Key: HIVE-26267
> URL: https://issues.apache.org/jira/browse/HIVE-26267
> Project: Hive
> Issue Type: Bug
>Reporter: László Végh
>Assignee: László Végh
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The assembled prepared statement in
> {code:java}
> org.apache.hadoop.hive.metastore.txn.TxnHandler#compact{code}
> does not work for Postgres DB.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
