[
https://issues.apache.org/jira/browse/HIVE-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17063709#comment-17063709
]
David Mollitor commented on HIVE-23005:
---------------------------------------
{quote}fix the initial fetchsize
{quote}
You'll have to be more specific there. After discussing it, I thought we were
good with the current solution. I will add documentation in a subsequent task
if you can provide me some pointers on where/how to do that.
However, the plan is, once this patch is pushed, in order of precedence, the
fetch size is determined this way :
# Fetch size is set based on what is received from HS2
({{hive.server2.thrift.resultset.default.fetch.size}}) during the client
session open sequence
# Fetch size is set in the JDBC connection string (not well documented:
{{jdbc:hive2://localhost:10000;fetchSize=100}})
# Fetch size is set by the application code via JDBC {{setFetchSize}} [1]
Keeping in mind that whatever the client requests for a fetch size will be
overruled on the HS2-side by {{hive.server2.thrift.resultset.max.fetch.size}}.
When a client requests a fetchSize larger than the max, a WARN message is
emitted into the HS2 logs for further investigation and to direct clients to
adjust their expectations (and configurations).
[1]
[https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-]
HIVE-22853 allows beeline to set the fetch size directly on the JDBC client
(#3). Therefore:
# If the beeline user does nothing, each query will use the fetch size
received from HS2
# If the beeline user sets the fetchSize in the JDBC connection string, each
query will use the fetch size specified there
# If the user wants to set the fetchSize in the session, they can with the
syntax: {{!set fetchSize xxx}}
## Setting a fetchSize of 0 will direct the driver to use the fetch size
provided from HS2
## Setting a fetchSize greater than 0 will set the driver fetch size to the
specified value
## Setting a fetchSize of -1 directs beeline to use the default JDBC default
behavior: use the connection string fetchSize and, if none is specified,
fallback to the fetch size specified by HS2 (this is the default beeline
fetchSize value)
## Setting a fetchSize of any other negative integer value is an error
> Consider Default JDBC Fetch Size From HS2
> -----------------------------------------
>
> Key: HIVE-23005
> URL: https://issues.apache.org/jira/browse/HIVE-23005
> Project: Hive
> Issue Type: Sub-task
> Components: JDBC
> Reporter: David Mollitor
> Assignee: David Mollitor
> Priority: Major
> Attachments: HIVE-23005.1.patch, HIVE-23005.2.patch,
> HIVE-23005.3.patch, HIVE-23005.4.patch, HIVE-23005.5.patch, HIVE-23005.6.patch
>
>
> # Create two fetch sizes: and "init" fetch size and a "default" fetch size.
> The "init" fetch size comes from the JDBC connection string _fetchSize_ (if
> present) and the "default" fetch size comes from
> _HIVE_SERVER2_THRIFT_RESULTSET_DEFAULT_FETCH_SIZE_ the server response to the
> open session request.
> # When a {{Statement}} is created, its starting fetch size is the "init"
> fetch size (may be 0)
> # Manually setting the fetch size on the {{Statement}} to 0, sets the fetch
> size to be the server default
> Setting to zero defaults to the server's instructed default and adheres to:
> bq. If the value specified is zero, then the hint is ignored. The default
> value is zero.
> That is to say, if the fetch size is 0, the default from the server is used,
> otherwise the user can pass a 'hint' and that will be the number of rows
> fetched instead.
> https://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
