[jira] [Work logged] (HIVE-24545) jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
[
https://issues.apache.org/jira/browse/HIVE-24545?focusedWorklogId=726068=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-726068
]
ASF GitHub Bot logged work on HIVE-24545:
-
Author: ASF GitHub Bot
Created on: 14/Feb/22 02:39
Start Date: 14/Feb/22 02:39
Worklog Time Spent: 10m
Work Description: github-actions[bot] closed pull request #1789:
URL: https://github.com/apache/hive/pull/1789
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 726068)
Time Spent: 1h 40m (was: 1.5h)
> jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
>
>
> Key: HIVE-24545
> URL: https://issues.apache.org/jira/browse/HIVE-24545
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> I found this while IOW on TPCDS 10TB:
> {code}
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. llap SUCCEEDED 4210 421000
> 0 362
> Reducer 2 .. llap SUCCEEDED10110100
> 0 2
> Reducer 3 .. llap SUCCEEDED 1009 100900
> 0 1
> --
> VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 12613.62 s
> --
> 20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
> than Integer.MAX_VALUE
> {code}
> my scenario was:
> {code}
> set hive.exec.max.dynamic.partitions=2000;
> drop table if exists test_sales_2;
> create table test_sales_2 like
> tpcds_bin_partitioned_acid_orc_1.store_sales;
> insert overwrite table test_sales_2 select * from
> tpcds_bin_partitioned_acid_orc_1.store_sales where ss_sold_date_sk >
> 2451868;
> {code}
> regarding affected row numbers:
> {code}
> select count(*) from tpcds_bin_partitioned_acid_orc_1.store_sales where
> ss_sold_date_sk > 2451868;
> +--+
> | _c0 |
> +--+
> | 12287871907 |
> +--+
> {code}
> I guess we should switch to long
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-24545) jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
[
https://issues.apache.org/jira/browse/HIVE-24545?focusedWorklogId=725862=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-725862
]
ASF GitHub Bot logged work on HIVE-24545:
-
Author: ASF GitHub Bot
Created on: 14/Feb/22 00:13
Start Date: 14/Feb/22 00:13
Worklog Time Spent: 10m
Work Description: github-actions[bot] closed pull request #1789:
URL: https://github.com/apache/hive/pull/1789
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 725862)
Time Spent: 1.5h (was: 1h 20m)
> jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
>
>
> Key: HIVE-24545
> URL: https://issues.apache.org/jira/browse/HIVE-24545
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> I found this while IOW on TPCDS 10TB:
> {code}
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. llap SUCCEEDED 4210 421000
> 0 362
> Reducer 2 .. llap SUCCEEDED10110100
> 0 2
> Reducer 3 .. llap SUCCEEDED 1009 100900
> 0 1
> --
> VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 12613.62 s
> --
> 20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
> than Integer.MAX_VALUE
> {code}
> my scenario was:
> {code}
> set hive.exec.max.dynamic.partitions=2000;
> drop table if exists test_sales_2;
> create table test_sales_2 like
> tpcds_bin_partitioned_acid_orc_1.store_sales;
> insert overwrite table test_sales_2 select * from
> tpcds_bin_partitioned_acid_orc_1.store_sales where ss_sold_date_sk >
> 2451868;
> {code}
> regarding affected row numbers:
> {code}
> select count(*) from tpcds_bin_partitioned_acid_orc_1.store_sales where
> ss_sold_date_sk > 2451868;
> +--+
> | _c0 |
> +--+
> | 12287871907 |
> +--+
> {code}
> I guess we should switch to long
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-24545) jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
[
https://issues.apache.org/jira/browse/HIVE-24545?focusedWorklogId=721601=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-721601
]
ASF GitHub Bot logged work on HIVE-24545:
-
Author: ASF GitHub Bot
Created on: 06/Feb/22 08:48
Start Date: 06/Feb/22 08:48
Worklog Time Spent: 10m
Work Description: abstractdog opened a new pull request #1789:
URL: https://github.com/apache/hive/pull/1789
### What changes were proposed in this pull request?
We should use java.sql.getLargeUpdateCount() where it's possible.
User-facing case is beeline output.
### Why are the changes needed?
Because this can be confusing for the user on beeline output:
```
20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
than Integer.MAX_VALUE
```
### Does this PR introduce _any_ user-facing change?
Yes, beeline is supposed to return row numbers > Integer.MAX_VALUE properly.
### How was this patch tested?
Not yet tested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 721601)
Time Spent: 1h 20m (was: 1h 10m)
> jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
>
>
> Key: HIVE-24545
> URL: https://issues.apache.org/jira/browse/HIVE-24545
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> I found this while IOW on TPCDS 10TB:
> {code}
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. llap SUCCEEDED 4210 421000
> 0 362
> Reducer 2 .. llap SUCCEEDED10110100
> 0 2
> Reducer 3 .. llap SUCCEEDED 1009 100900
> 0 1
> --
> VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 12613.62 s
> --
> 20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
> than Integer.MAX_VALUE
> {code}
> my scenario was:
> {code}
> set hive.exec.max.dynamic.partitions=2000;
> drop table if exists test_sales_2;
> create table test_sales_2 like
> tpcds_bin_partitioned_acid_orc_1.store_sales;
> insert overwrite table test_sales_2 select * from
> tpcds_bin_partitioned_acid_orc_1.store_sales where ss_sold_date_sk >
> 2451868;
> {code}
> regarding affected row numbers:
> {code}
> select count(*) from tpcds_bin_partitioned_acid_orc_1.store_sales where
> ss_sold_date_sk > 2451868;
> +--+
> | _c0 |
> +--+
> | 12287871907 |
> +--+
> {code}
> I guess we should switch to long
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-24545) jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
[
https://issues.apache.org/jira/browse/HIVE-24545?focusedWorklogId=721550=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-721550
]
ASF GitHub Bot logged work on HIVE-24545:
-
Author: ASF GitHub Bot
Created on: 06/Feb/22 00:52
Start Date: 06/Feb/22 00:52
Worklog Time Spent: 10m
Work Description: github-actions[bot] closed pull request #1789:
URL: https://github.com/apache/hive/pull/1789
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 721550)
Time Spent: 1h 10m (was: 1h)
> jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
>
>
> Key: HIVE-24545
> URL: https://issues.apache.org/jira/browse/HIVE-24545
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> I found this while IOW on TPCDS 10TB:
> {code}
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. llap SUCCEEDED 4210 421000
> 0 362
> Reducer 2 .. llap SUCCEEDED10110100
> 0 2
> Reducer 3 .. llap SUCCEEDED 1009 100900
> 0 1
> --
> VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 12613.62 s
> --
> 20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
> than Integer.MAX_VALUE
> {code}
> my scenario was:
> {code}
> set hive.exec.max.dynamic.partitions=2000;
> drop table if exists test_sales_2;
> create table test_sales_2 like
> tpcds_bin_partitioned_acid_orc_1.store_sales;
> insert overwrite table test_sales_2 select * from
> tpcds_bin_partitioned_acid_orc_1.store_sales where ss_sold_date_sk >
> 2451868;
> {code}
> regarding affected row numbers:
> {code}
> select count(*) from tpcds_bin_partitioned_acid_orc_1.store_sales where
> ss_sold_date_sk > 2451868;
> +--+
> | _c0 |
> +--+
> | 12287871907 |
> +--+
> {code}
> I guess we should switch to long
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-24545) jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
[
https://issues.apache.org/jira/browse/HIVE-24545?focusedWorklogId=717407=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-717407
]
ASF GitHub Bot logged work on HIVE-24545:
-
Author: ASF GitHub Bot
Created on: 29/Jan/22 00:11
Start Date: 29/Jan/22 00:11
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on pull request #1789:
URL: https://github.com/apache/hive/pull/1789#issuecomment-1024761518
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the d...@hive.apache.org list if the patch is in
need of reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 717407)
Time Spent: 1h (was: 50m)
> jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
>
>
> Key: HIVE-24545
> URL: https://issues.apache.org/jira/browse/HIVE-24545
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> I found this while IOW on TPCDS 10TB:
> {code}
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. llap SUCCEEDED 4210 421000
> 0 362
> Reducer 2 .. llap SUCCEEDED10110100
> 0 2
> Reducer 3 .. llap SUCCEEDED 1009 100900
> 0 1
> --
> VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 12613.62 s
> --
> 20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
> than Integer.MAX_VALUE
> {code}
> my scenario was:
> {code}
> set hive.exec.max.dynamic.partitions=2000;
> drop table if exists test_sales_2;
> create table test_sales_2 like
> tpcds_bin_partitioned_acid_orc_1.store_sales;
> insert overwrite table test_sales_2 select * from
> tpcds_bin_partitioned_acid_orc_1.store_sales where ss_sold_date_sk >
> 2451868;
> {code}
> regarding affected row numbers:
> {code}
> select count(*) from tpcds_bin_partitioned_acid_orc_1.store_sales where
> ss_sold_date_sk > 2451868;
> +--+
> | _c0 |
> +--+
> | 12287871907 |
> +--+
> {code}
> I guess we should switch to long
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-24545) jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
[
https://issues.apache.org/jira/browse/HIVE-24545?focusedWorklogId=687338=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-687338
]
ASF GitHub Bot logged work on HIVE-24545:
-
Author: ASF GitHub Bot
Created on: 29/Nov/21 14:18
Start Date: 29/Nov/21 14:18
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #1789:
URL: https://github.com/apache/hive/pull/1789#discussion_r758402670
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -587,6 +587,26 @@ public int getUpdateCount() throws SQLException {
return (int) numModifiedRows;
}
+ @Override
+ public long getLargeUpdateCount() throws SQLException {
+checkConnection("getLargeUpdateCount");
+/**
+ * Poll on the operation status, till the operation is complete. We want
to ensure that since a
+ * client might end up using executeAsync and then call this to check if
the query run is
+ * finished.
+ */
+long numModifiedRows = -1L;
+TGetOperationStatusResp resp = waitForOperationToComplete();
+if (resp != null) {
+ numModifiedRows = resp.getNumModifiedRows();
+}
+if (numModifiedRows == -1L || numModifiedRows > Long.MAX_VALUE) {
+ LOG.warn("Invalid number of updated rows: {}", numModifiedRows);
+ return -1;
Review comment:
I'm not sure if returning `-1` is the best way to signal this
problems... especially in the old `getUpdateCount` method
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -587,6 +587,26 @@ public int getUpdateCount() throws SQLException {
return (int) numModifiedRows;
}
+ @Override
+ public long getLargeUpdateCount() throws SQLException {
+checkConnection("getLargeUpdateCount");
+/**
+ * Poll on the operation status, till the operation is complete. We want
to ensure that since a
+ * client might end up using executeAsync and then call this to check if
the query run is
+ * finished.
+ */
+long numModifiedRows = -1L;
+TGetOperationStatusResp resp = waitForOperationToComplete();
+if (resp != null) {
+ numModifiedRows = resp.getNumModifiedRows();
+}
+if (numModifiedRows == -1L || numModifiedRows > Long.MAX_VALUE) {
Review comment:
is `-2` valid?
we could reuse the newly implemented method in the old `getUpdateCount` to
reduce code duplication
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 687338)
Time Spent: 50m (was: 40m)
> jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
>
>
> Key: HIVE-24545
> URL: https://issues.apache.org/jira/browse/HIVE-24545
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> I found this while IOW on TPCDS 10TB:
> {code}
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. llap SUCCEEDED 4210 421000
> 0 362
> Reducer 2 .. llap SUCCEEDED10110100
> 0 2
> Reducer 3 .. llap SUCCEEDED 1009 100900
> 0 1
> --
> VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 12613.62 s
> --
> 20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
> than Integer.MAX_VALUE
> {code}
> my scenario was:
> {code}
> set hive.exec.max.dynamic.partitions=2000;
> drop table if exists test_sales_2;
> create table test_sales_2 like
> tpcds_bin_partitioned_acid_orc_1.store_sales;
> insert overwrite table test_sales_2 select * from
> tpcds_bin_partitioned_acid_orc_1.store_sales where ss_sold_date_sk >
> 2451868;
> {code}
> regarding affected row numbers:
> {code}
> select count(*) from tpcds_bin_partitioned_acid_orc_1.store_sales where
> ss_sold_date_sk > 2451868;
>
[jira] [Work logged] (HIVE-24545) jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
[
https://issues.apache.org/jira/browse/HIVE-24545?focusedWorklogId=684578=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-684578
]
ASF GitHub Bot logged work on HIVE-24545:
-
Author: ASF GitHub Bot
Created on: 22/Nov/21 09:07
Start Date: 22/Nov/21 09:07
Worklog Time Spent: 10m
Work Description: abstractdog opened a new pull request #1789:
URL: https://github.com/apache/hive/pull/1789
### What changes were proposed in this pull request?
We should use java.sql.getLargeUpdateCount() where it's possible.
User-facing case is beeline output.
### Why are the changes needed?
Because this can be confusing for the user on beeline output:
```
20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
than Integer.MAX_VALUE
```
### Does this PR introduce _any_ user-facing change?
Yes, beeline is supposed to return row numbers > Integer.MAX_VALUE properly.
### How was this patch tested?
Not yet tested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 684578)
Time Spent: 40m (was: 0.5h)
> jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
>
>
> Key: HIVE-24545
> URL: https://issues.apache.org/jira/browse/HIVE-24545
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I found this while IOW on TPCDS 10TB:
> {code}
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. llap SUCCEEDED 4210 421000
> 0 362
> Reducer 2 .. llap SUCCEEDED10110100
> 0 2
> Reducer 3 .. llap SUCCEEDED 1009 100900
> 0 1
> --
> VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 12613.62 s
> --
> 20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
> than Integer.MAX_VALUE
> {code}
> my scenario was:
> {code}
> set hive.exec.max.dynamic.partitions=2000;
> drop table if exists test_sales_2;
> create table test_sales_2 like
> tpcds_bin_partitioned_acid_orc_1.store_sales;
> insert overwrite table test_sales_2 select * from
> tpcds_bin_partitioned_acid_orc_1.store_sales where ss_sold_date_sk >
> 2451868;
> {code}
> regarding affected row numbers:
> {code}
> select count(*) from tpcds_bin_partitioned_acid_orc_1.store_sales where
> ss_sold_date_sk > 2451868;
> +--+
> | _c0 |
> +--+
> | 12287871907 |
> +--+
> {code}
> I guess we should switch to long
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-24545) jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
[
https://issues.apache.org/jira/browse/HIVE-24545?focusedWorklogId=558835=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-558835
]
ASF GitHub Bot logged work on HIVE-24545:
-
Author: ASF GitHub Bot
Created on: 27/Feb/21 00:48
Start Date: 27/Feb/21 00:48
Worklog Time Spent: 10m
Work Description: github-actions[bot] closed pull request #1789:
URL: https://github.com/apache/hive/pull/1789
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 558835)
Time Spent: 0.5h (was: 20m)
> jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
>
>
> Key: HIVE-24545
> URL: https://issues.apache.org/jira/browse/HIVE-24545
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> I found this while IOW on TPCDS 10TB:
> {code}
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. llap SUCCEEDED 4210 421000
> 0 362
> Reducer 2 .. llap SUCCEEDED10110100
> 0 2
> Reducer 3 .. llap SUCCEEDED 1009 100900
> 0 1
> --
> VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 12613.62 s
> --
> 20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
> than Integer.MAX_VALUE
> {code}
> my scenario was:
> {code}
> set hive.exec.max.dynamic.partitions=2000;
> drop table if exists test_sales_2;
> create table test_sales_2 like
> tpcds_bin_partitioned_acid_orc_1.store_sales;
> insert overwrite table test_sales_2 select * from
> tpcds_bin_partitioned_acid_orc_1.store_sales where ss_sold_date_sk >
> 2451868;
> {code}
> regarding affected row numbers:
> {code}
> select count(*) from tpcds_bin_partitioned_acid_orc_1.store_sales where
> ss_sold_date_sk > 2451868;
> +--+
> | _c0 |
> +--+
> | 12287871907 |
> +--+
> {code}
> I guess we should switch to long
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24545) jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
[
https://issues.apache.org/jira/browse/HIVE-24545?focusedWorklogId=555008=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-555008
]
ASF GitHub Bot logged work on HIVE-24545:
-
Author: ASF GitHub Bot
Created on: 20/Feb/21 00:46
Start Date: 20/Feb/21 00:46
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on pull request #1789:
URL: https://github.com/apache/hive/pull/1789#issuecomment-782482366
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the d...@hive.apache.org list if the patch is in
need of reviews.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 555008)
Time Spent: 20m (was: 10m)
> jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
>
>
> Key: HIVE-24545
> URL: https://issues.apache.org/jira/browse/HIVE-24545
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I found this while IOW on TPCDS 10TB:
> {code}
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. llap SUCCEEDED 4210 421000
> 0 362
> Reducer 2 .. llap SUCCEEDED10110100
> 0 2
> Reducer 3 .. llap SUCCEEDED 1009 100900
> 0 1
> --
> VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 12613.62 s
> --
> 20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
> than Integer.MAX_VALUE
> {code}
> my scenario was:
> {code}
> set hive.exec.max.dynamic.partitions=2000;
> drop table if exists test_sales_2;
> create table test_sales_2 like
> tpcds_bin_partitioned_acid_orc_1.store_sales;
> insert overwrite table test_sales_2 select * from
> tpcds_bin_partitioned_acid_orc_1.store_sales where ss_sold_date_sk >
> 2451868;
> {code}
> regarding affected row numbers:
> {code}
> select count(*) from tpcds_bin_partitioned_acid_orc_1.store_sales where
> ss_sold_date_sk > 2451868;
> +--+
> | _c0 |
> +--+
> | 12287871907 |
> +--+
> {code}
> I guess we should switch to long
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24545) jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
[
https://issues.apache.org/jira/browse/HIVE-24545?focusedWorklogId=524961=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-524961
]
ASF GitHub Bot logged work on HIVE-24545:
-
Author: ASF GitHub Bot
Created on: 16/Dec/20 11:00
Start Date: 16/Dec/20 11:00
Worklog Time Spent: 10m
Work Description: abstractdog opened a new pull request #1789:
URL: https://github.com/apache/hive/pull/1789
### What changes were proposed in this pull request?
We should use java.sql.getLargeUpdateCount() where it's possible.
User-facing case is beeline output.
### Why are the changes needed?
Because this can be confusing for the user on beeline output:
```
20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
than Integer.MAX_VALUE
```
### Does this PR introduce _any_ user-facing change?
Yes, beeline is supposed to return row numbers > Integer.MAX_VALUE properly.
### How was this patch tested?
Not yet tested.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 524961)
Remaining Estimate: 0h
Time Spent: 10m
> jdbc.HiveStatement: Number of rows is greater than Integer.MAX_VALUE
>
>
> Key: HIVE-24545
> URL: https://issues.apache.org/jira/browse/HIVE-24545
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I found this while IOW on TPCDS 10TB:
> {code}
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. llap SUCCEEDED 4210 421000
> 0 362
> Reducer 2 .. llap SUCCEEDED10110100
> 0 2
> Reducer 3 .. llap SUCCEEDED 1009 100900
> 0 1
> --
> VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 12613.62 s
> --
> 20/12/16 01:37:36 [main]: WARN jdbc.HiveStatement: Number of rows is greater
> than Integer.MAX_VALUE
> {code}
> my scenario was:
> {code}
> set hive.exec.max.dynamic.partitions=2000;
> drop table if exists test_sales_2;
> create table test_sales_2 like
> tpcds_bin_partitioned_acid_orc_1.store_sales;
> insert overwrite table test_sales_2 select * from
> tpcds_bin_partitioned_acid_orc_1.store_sales where ss_sold_date_sk >
> 2451868;
> {code}
> regarding affected row numbers:
> {code}
> select count(*) from tpcds_bin_partitioned_acid_orc_1.store_sales where
> ss_sold_date_sk > 2451868;
> +--+
> | _c0 |
> +--+
> | 12287871907 |
> +--+
> {code}
> I guess we should switch to long
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
