[
https://issues.apache.org/jira/browse/SQOOP-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274707#comment-14274707
]
Veena Basavaraj commented on SQOOP-1989:
----------------------------------------
[~jarcec] your thoughts on this? it would be good to optimize and not always
call the matcher code.
> IDF optimization to store object array in memory ( so matching is faster)
> -------------------------------------------------------------------------
>
> Key: SQOOP-1989
> URL: https://issues.apache.org/jira/browse/SQOOP-1989
> Project: Sqoop
> Issue Type: Bug
> Reporter: Veena Basavaraj
> Fix For: 2.0.0
>
>
> according to the IDF api, we never cache or store the csv text/ not object
> array representation in memory, we only store the native format as below
> {code}
> **
> * Get one row of data.
> *
> * @return - One row of data, represented in the internal/native format of
> * the intermediate data format implementation.
> */
> public T getData() {
> return data;
> }
> {code}
> But the matcher code in SqoopWritable always calls the setObjectData and
> getObjectData on every row in the data, which mean we exercise this call no
> matter what the native format is
> toIDF.setObjectData(matcher.getMatchingData(fromIDF.getObjectData()));
> So should we not store the object array in memory?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
