[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-02-07 Thread Naveen Gangam (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032506#comment-17032506
 ] 

Naveen Gangam commented on HIVE-22853:
--

Published to RB at https://reviews.apache.org/r/72101/

[~thejas]

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-02-21 Thread David Mollitor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17042352#comment-17042352
 ] 

David Mollitor commented on HIVE-22853:
---

[~ngangam] What is the value of being able to change this value in the middle 
of a session?  What is the use case there?  Perhaps it's better to simply 
prohibit users from changing the value mid-session.

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-02-24 Thread Naveen Gangam (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043651#comment-17043651
 ] 

Naveen Gangam commented on HIVE-22853:
--

[~belugabehr] This isn't just a beeline thing. JDBC spec allows a fetchsize to 
be set at the statement level. So HS2 has to support accepting user input for 
each FetchResults call. Inherently, beeline should allow it to be changed 
within session as well, between queries if the user chooses so.
While I agree that this value, for optimal performance, is more dependent on 
the resources available(primarily Xmx) in beeline than on the result size of a 
particular query, being able to set it within a session, provides the ability 
to tune without having to drop the session.
Hope this makes sense.

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-02-24 Thread Yongzhi Chen (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043759#comment-17043759
 ] 

Yongzhi Chen commented on HIVE-22853:
-

Patch 3 LGTM   +1 Pending tests.

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-02-24 Thread David Mollitor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043763#comment-17043763
 ] 

David Mollitor commented on HIVE-22853:
---

OK.  I just you that there is value here, I'm not sure I see it.

 If you want to do this, I don't think the current approach is correct.
 

[https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-]

{quote}
If the value specified is zero, then the hint is ignored. The default value is 
zero.

SQLException - the condition rows >= 0 is not satisfied.
{quote}
 
So, I think you should adhere more closely to this spec to avoid ambiguity and 
you don't need to check for a valid value.  Just always set the fetch size on 
the statement.  Also think about what is the behavior if the user attempts to 
set something less than zero?  Right now the patch ignores the value, I'm not 
sure that's the best approach.   Better to validate the user supplied value and 
log an error if the value is invalid otherwise user will not know that their 
change didn't take affect.

Also, the proposed code is too brittle.  I would like to see this actually 
parse the statement and then look for the value instead of just this simple, 
brittle string match.  Would HiveServer2 accept the following?

{code:none}
set hive.server2.thrift.resultset.default.fetch.size=1;
set hive.server2.thrift.resultset.default.fetch.size=1;
{code}

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-02-24 Thread Naveen Gangam (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043944#comment-17043944
 ] 

Naveen Gangam commented on HIVE-22853:
--

[~belugabehr] Thanks for the review.
Fair point regarding the default. But this needs more thought, I think. Having 
a no-limit read when dealing with large scale data, worries me. It could 
potentially crash both services.
We can fail the set command if value is less than 0. The question still remains 
if ZERO should be permitted and honored. [~ychena] Do you have any thoughts on 
having a no-limit fetch for 0?

>From the hive documentation, 
{noformat}
set =

Sets the value of a particular configuration variable (key).
Note: If you misspell the variable name, the CLI will not show an error.
{noformat}

I do not know if the following syntax is legal or works currently. I was going 
by what has been stated in the documentation. But if you know that users have 
been using alternate syntax such as these, it should be easy enough to 
incorporate this change.
{noformat}
set hive.server2.thrift.resultset.default.fetch.size=1;
set hive.server2.thrift.resultset.default.fetch.size =1
set hive.server2.thrift.resultset.default.fetch.size = 1;
{noformat}



> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-02-24 Thread Yongzhi Chen (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043983#comment-17043983
 ] 

Yongzhi Chen commented on HIVE-22853:
-

[~ngangam], if we follow the same spec as Oracle, we should treat less than 0 
as invalid, and default to 0 and treat zero as no-limit. 

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-02-24 Thread Hive QA (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17044108#comment-17044108
 ] 

Hive QA commented on HIVE-22853:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
24s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
31s{color} | {color:blue} jdbc in master has 16 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
32s{color} | {color:blue} beeline in master has 48 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} jdbc: The patch generated 0 new + 27 unchanged - 1 
fixed = 27 total (was 28) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
13s{color} | {color:red} beeline: The patch generated 2 new + 67 unchanged - 0 
fixed = 69 total (was 67) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 
14s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 51s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20811/dev-support/hive-personality.sh
 |
| git revision | master / 0767c5d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20811/yetus/diff-checkstyle-beeline.txt
 |
| asflicense | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20811/yetus/patch-asflicense-problems.txt
 |
| modules | C: jdbc beeline U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20811/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should defaul

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-02-24 Thread Hive QA (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17044124#comment-17044124
 ] 

Hive QA commented on HIVE-22853:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12994369/HIVE-22853.3.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 18057 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20811/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20811/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20811/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12994369 - PreCommit-HIVE-Build

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-02-25 Thread David Mollitor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17044466#comment-17044466
 ] 

David Mollitor commented on HIVE-22853:
---

{quote}
if we follow the same spec as Oracle, we should treat less than 0 as invalid, 
and default to 0 and treat zero as no-limit.
{quote}

I do not think the is correct.  From the docs: _If the value specified is zero, 
then the hint is ignored._

I think this means simply that the driver can use whatever default value it 
wants since the client application has not provided any kind of hint.

So, actually the 'default' value should be 0 in BeeLine.java and should be 
treated as 1000 (current default) in the Driver itself... this behavior is 
correctly implemented in the driver:

https://github.com/apache/hive/blob/037eacea46371015a7f9894c5a9ccfb9708d5c56/jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java#L811

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-02-25 Thread David Mollitor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17044468#comment-17044468
 ] 

David Mollitor commented on HIVE-22853:
---

Same thing in HiveConnection.java, the fetchSize should default to 0.

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-02-28 Thread Hive QA (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17048218#comment-17048218
 ] 

Hive QA commented on HIVE-22853:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12994970/HIVE-22853.4.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20883/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20883/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20883/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-02-29 06:35:15.534
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-20883/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-02-29 06:35:15.537
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 1fc351b HIVE-22925: Implement TopNKeyFilter efficiency check 
(Attila Magyar, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 1fc351b HIVE-22925: Implement TopNKeyFilter efficiency check 
(Attila Magyar, reviewed by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-02-29 06:35:24.952
+ rm -rf ../yetus_PreCommit-HIVE-Build-20883
+ mkdir ../yetus_PreCommit-HIVE-Build-20883
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-20883
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20883/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: patch failed: beeline/src/java/org/apache/hive/beeline/Commands.java:52
Falling back to three-way merge...
Applied patch to 'beeline/src/java/org/apache/hive/beeline/Commands.java' with 
conflicts.
Going to apply patch with: git apply -p0
error: patch failed: beeline/src/java/org/apache/hive/beeline/Commands.java:52
Falling back to three-way merge...
Applied patch to 'beeline/src/java/org/apache/hive/beeline/Commands.java' with 
conflicts.
U beeline/src/java/org/apache/hive/beeline/Commands.java
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-20883
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12994970 - PreCommit-HIVE-Build

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, 
> HIVE-22853.4.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-03-01 Thread Naveen Gangam (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17048778#comment-17048778
 ] 

Naveen Gangam commented on HIVE-22853:
--

Looks like there are changes on master. I am attaching a rebased version of the 
patch.

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, 
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-03-02 Thread Hive QA (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17048861#comment-17048861
 ] 

Hive QA commented on HIVE-22853:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
51s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
 0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
40s{color} | {color:blue} service in master has 51 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
30s{color} | {color:blue} jdbc in master has 16 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
31s{color} | {color:blue} beeline in master has 48 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} service: The patch generated 0 new + 27 unchanged - 
1 fixed = 27 total (was 28) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
11s{color} | {color:red} jdbc: The patch generated 3 new + 43 unchanged - 0 
fixed = 46 total (was 43) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
13s{color} | {color:red} beeline: The patch generated 2 new + 67 unchanged - 0 
fixed = 69 total (was 67) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m 
49s{color} | {color:red} service generated 1 new + 51 unchanged - 0 fixed = 52 
total (was 51) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 
15s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 43s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:service |
|  |  Dead store to sessionConf in 
org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(TOpenSessionReq)
  At 
ThriftCLIService.java:org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(TOpenSessionReq)
  At ThriftCLIService.java:[line 328] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20906/dev-support/hive-personality.sh
 |
| git revision | master / f6bae78 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20906/yetus/diff-checkstyle-jdbc.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20906/yetus/diff-checkstyle-beeline.txt
 |
| findbugs | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20906/yetus/new-findbugs-service.html
 |
| asflicense | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20906/yetus/patch-asflicense-problems.txt
 |
| modules | C: service jdbc beeline U: . |
| Console output | 
http://104.198.109.2

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-03-02 Thread David Mollitor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17049271#comment-17049271
 ] 

David Mollitor commented on HIVE-22853:
---

{code:java|title=BeeLine.java}
+  public int fetchSize = -1;

+  public void setFetchSize(int fetchRows) {
+if (fetchRows <= 0) { // if set to 0 or less, use server side default
+  this.fetchSize = Utils.getServerDefaultFetchSize();
+} else {
+  this.fetchSize = fetchRows;
+}
+  }

+
 if (getOpts().timeout > -1) {
   stmnt.setQueryTimeout(getOpts().timeout);
 }
 if (signalHandler != null) {
   signalHandler.setStatement(stmnt);
 }
+if (fetchSize > -1) {
+  stmnt.setFetchSize(fetchSize);
+}
{code}

# "-1" is an invalid value for fetch size, please make the default "0" so that 
it lines up with the JDBC specs.  The {{setFetchSize}} method makes sure the 
value is always valid.
# Remove the blank line change to minimize the patch
# Do not check for a valid fetchSize, just set it 
{{stmnt.setFetchSize(fetchSize);}}}

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, 
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-03-02 Thread David Mollitor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17049276#comment-17049276
 ] 

David Mollitor commented on HIVE-22853:
---

{code:java}
Pattern setPattern = 
Pattern.compile("(\\s*)(set)(\\s+)(\\S+)(\\s*)=(\\s*)(-?\\d+)(\\s*)(;*)");
{code}

Do not bother capturing 7 groups.  Just capture the two groups that matter:

{code:java}
Pattern setPattern = Pattern.compile("\\s*set\\s+(\\S+)\\s*=\\s*(-?\\d+)\\s*;*);
{code}

Do not worry about beginning and trailing spaces:

{code:java}
Pattern setPattern = Pattern.compile("set\\s+(\\S+)\\s*=\\s*(-?\\d+)\\s*;);
Matcher matcher = setPattern.matcher(sql.toLowerCase().trim());
{code}

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, 
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-03-02 Thread David Mollitor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17049292#comment-17049292
 ] 

David Mollitor commented on HIVE-22853:
---

{code:java|title=Utils.java}
+  static int serverDefaultFetchSize = -1;
{code}

Please do not store state in a utility class.

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, 
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-03-09 Thread Hive QA (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055575#comment-17055575
 ] 

Hive QA commented on HIVE-22853:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
40s{color} | {color:blue} service in master has 51 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
29s{color} | {color:blue} jdbc in master has 16 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
28s{color} | {color:blue} beeline in master has 48 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
41s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
13s{color} | {color:green} service: The patch generated 0 new + 27 unchanged - 
1 fixed = 27 total (was 28) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
10s{color} | {color:red} jdbc: The patch generated 2 new + 33 unchanged - 0 
fixed = 35 total (was 33) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
12s{color} | {color:red} beeline: The patch generated 2 new + 67 unchanged - 0 
fixed = 69 total (was 67) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m 
48s{color} | {color:red} service generated 1 new + 51 unchanged - 0 fixed = 52 
total (was 51) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 
13s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 38s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:service |
|  |  Dead store to sessionConf in 
org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(TOpenSessionReq)
  At 
ThriftCLIService.java:org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(TOpenSessionReq)
  At ThriftCLIService.java:[line 328] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21045/dev-support/hive-personality.sh
 |
| git revision | master / a003428 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-21045/yetus/diff-checkstyle-jdbc.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-21045/yetus/diff-checkstyle-beeline.txt
 |
| findbugs | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-21045/yetus/new-findbugs-service.html
 |
| asflicense | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-21045/yetus/patch-asflicense-problems.txt
 |
| modules | C: service jdbc beeline U: . |
| Console output | 
http://104.198.109.2

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-03-09 Thread Hive QA (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055613#comment-17055613
 ] 

Hive QA commented on HIVE-22853:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996200/HIVE-22853.6.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 18093 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestMetastoreHousekeepingLeaderEmptyConfig.testHouseKeepingThreadExistence
 (batchId=252)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testFetchSize (batchId=293)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/21045/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21045/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21045/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12996200 - PreCommit-HIVE-Build

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, 
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-03-10 Thread David Mollitor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055945#comment-17055945
 ] 

David Mollitor commented on HIVE-22853:
---

[~ngangam] I think you're going to want to {{trim()}} this too:

{code:java}
Matcher matcher = setPattern.matcher(sql.toLowerCase());
{code}

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, 
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22853) Beeline should use HS2 server defaults for fetchSize

2020-03-10 Thread David Mollitor (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056022#comment-17056022
 ] 

David Mollitor commented on HIVE-22853:
---

[~ngangam] OK. I put together a patch that I think addresses part of this JIRA, 
but I can't seem to attach it for some reason. Maybe because it's assigned to 
you and in "patch" state.  I'm going to open a sub-task.

This JIRA is really two parts and should probably be 2 JIRAs:
 # Update Hive JDBC to use server default fetch size
 # Update Beeline to overwrite the fetch size when supplied on the command line

I do worry #2 is too specific to Beeline. If this is a good feature to have, it 
should be included in the JDBC library and not just hacked into beeline in this 
adhoc manner, though implementing it well would be tricky and probably not 
worth the reward.

My patch does the following:
 # Create two fetch sizes: and "init" fetch size and a "default" fetch size. 
The "init" fetch size comes from the JDBC connection string _fetchSize_ (if 
present) and the "default" fetch size comes from the server response to the 
open session request.
 # When a connection is made, it's starting fetch size is the "init" fetch size 
(may be 0)
 # Manually setting the fetch size to 0, sets the fetch size to be the server 
default

This is a sensible thing to do any for the JDBC library and any application 
that uses JDBC benefits, not just beeline (which has limited use).

For beeline-specific features, I think it's wonky to be parsing (regexing) a 
SQL statement and then trying to decipher a HiveServer setting. I would much 
rather see something specific to beeline that doesn't need to change if the HS2 
or JDBC changes.

Let there be a Hive specific feature and a Beeline specific feature:

{code:java}
set hive.server2.thrift.resultset.default.fetch.size=200;
!set fetchsize 200
{code}
[http://sqlline.sourceforge.net/#sect_command_properties]

> Beeline should use HS2 server defaults for fetchSize
> 
>
> Key: HIVE-22853
> URL: https://issues.apache.org/jira/browse/HIVE-22853
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22853.2.patch, HIVE-22853.3.patch, 
> HIVE-22853.4.patch, HIVE-22853.5.patch, HIVE-22853.6.patch, HIVE-22853.patch
>
>
> Currently beeline uses a hard coded default of 1000 rows for fetchSize. This 
> default value is different from what the server has set. While the beeline 
> user can reset the value via set command, its cumbersome to change the 
> workloads.
> Rather it should default to the server-side value and set should be used to 
> override within the session.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)