[
https://issues.apache.org/jira/browse/TRAFODION-2949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416027#comment-16416027
]
Sandhya Sundaresan commented on TRAFODION-2949:
-----------------------------------------------
Making a few fixes in this area.
# LOB internal descriptor tables were treated as "special" tables and
prevented caching of NATable information. So ever insert into a lob column that
needed to update the LOB internal descriptor tables had to be reloaded each
time. Removing that restriction. This signinifantyl improves insert/update
operations.
# Fixed an issue where only one type of internal table was detected as LOB
table and caused query cache to not cache statements related to these tables.
LOBDescChunks was not recognized. Added that.
# Made the units of the LOB Max size , LOB size consistent in the compiletime
and runtime structures.Now Describe returns the units in bytes.
# Disabled doing GC during DML operations by default. There will be a separate
utility for doing this or it can be enabled by setting CQD LOB_GC_LIMIT_SIZE to
something other than -1.
# Increased the rowset size limit checks to allow for batch operations to
support input rows as large as 16MB.
> LOB: Address performance issues related to inserts in tables containing LOB
> columns
> -----------------------------------------------------------------------------------
>
> Key: TRAFODION-2949
> URL: https://issues.apache.org/jira/browse/TRAFODION-2949
> Project: Apache Trafodion
> Issue Type: Improvement
> Reporter: Sandhya Sundaresan
> Assignee: Sandhya Sundaresan
> Priority: Major
>
> # Investigate bottlenecks in the insert and update code paths.
> A couple of bottlenecks identified initially were hdfsWrite and the LOBCLi
> call.
> # Investigate if lib hdfs write operation is signifincantly slower than a
> direct JNI call. If so consider refactoring code being done for hive inserts
> and reuse for LOB writes as well. Also investigate use of libhdfs3 , a
> lightweight interface by Pivotal for Hawk ,that doesn’t use JNI at all.
> # Investigate if LOB Cli calls that do prepare and execute of statements to
> insert/update the LOB descriptor tables are being cached and reused when
> multiple LOB inserts are being done in the same session.
> # Investigate external LOBs as a solution for faster inserts/extracts for
> usage compared to internal LOBs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
