[
https://issues.apache.org/jira/browse/HIVE-14170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15384559#comment-15384559
]
Tao Li edited comment on HIVE-14170 at 7/19/16 5:38 PM:
--------------------------------------------------------
[~stakiar] Another thinking is that we may improve the "buffered page" mode to
avoid OOM issue. For example, we can iterate through the whole result set once
to calculate the max column width (and without loading the result set into
memory). Then we iterate the result set again to print out. The pros is that it
requires minimal code change. The cons is that the latency should be higher
because we iterate the result set twice.
was (Author: taoli-hwx):
[~stakiar] Another thinking is that we may improve the "buffered page" mode to
avoid OOM issue. For example, we can iterate through the whole result set once
to calculate the max column width (and without loading the result set into
memory). Then we iterate the result set again to print out. The pros is that it
requires minimal code change. The cons is that the latency should be higher
because we iterate the result set twice.
> Beeline IncrementalRows should buffer rows and incrementally re-calculate
> width if TableOutputFormat is used
> ------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-14170
> URL: https://issues.apache.org/jira/browse/HIVE-14170
> Project: Hive
> Issue Type: Sub-task
> Components: Beeline
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Attachments: HIVE-14170.1.patch, HIVE-14170.2.patch
>
>
> If {{--incremental}} is specified in Beeline, rows are meant to be printed
> out immediately. However, if {{TableOutputFormat}} is used with this option
> the formatting can look really off.
> The reason is that {{IncrementalRows}} does not do a global calculation of
> the optimal width size for {{TableOutputFormat}} (it can't because it only
> sees one row at a time). The output of {{BufferedRows}} looks much better
> because it can do this global calculation.
> If {{--incremental}} is used, and {{TableOutputFormat}} is used, the width
> should be re-calculated every "x" rows ("x" can be configurable and by
> default it can be 1000).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
