[
https://issues.apache.org/jira/browse/CALCITE-2322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17371587#comment-17371587
]
Vladimir Sitnikov commented on CALCITE-2322:
--------------------------------------------
Just in case, the users rarely need "row count" when they configure fetch size.
In 99.42% of the cases what they need is to make sure fetch size is big enough
(to avoid too many roundtrips), and they want it small enough (to avoid out of
memory conditions).
So, in my opinion, what they really need is "fetch_size_bytes" rather than
"fetch_size_rows". Unfortunately, JDBC spec is broken there, and there's no API
for "fetch size bytes", however, the databases might support that (e.g. "please
fetch 10000 rows maximum, but stop as soon as you overflow 1 megabyte").
----
Suppose dynamic fetch would be implemented in the future. In other words,
Avatica could fetch more or fewer rows depending on the volume of the data
(e.g. to prevent too small or too large resultsets).
I guess it is more or less the direction many JDBC drivers are moving.
In that case, default_... makes it easier to reason: ok, this is a default
fetch size we start from, then it could adapt.
In other words: suppose you configure fetch_row_count=42, and you do not call
setFetchSize with Java API. Is the system allowed to fetch 1000 rows behind the
scenes?
My point is that default_fetch_row_count=42 (or default_fetch_size_rows or even
default_fetch_size_bytes) would keep the door open.
However, I do not insist on the naming since there is no single right answer.
----
As far as I know, Microsoft SQL Server JDBC driver supports only dynamic fetch
algorithm (I guess they just ignore setFetchSize calls).
> Add fetch size support to connection url and JDBC statement
> -----------------------------------------------------------
>
> Key: CALCITE-2322
> URL: https://issues.apache.org/jira/browse/CALCITE-2322
> Project: Calcite
> Issue Type: Improvement
> Components: avatica, core
> Affects Versions: 1.11.0
> Reporter: Kevin Minder
> Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Currently the remote driver defaults to hard coded fetch size of 100 rows.
> When a connection is operating in HTTP mode having such a small fetch size
> can add enormous overhead. This is especially true if TLS connections are
> used and made worse if each connection flows throw multiple proxies.
> Consider that 100K rows returned 100 rows at a time will make 1K HTTP POST
> requests. One might say that nobody should ever do that but some tools like
> Spotfire may end up doing this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
