[
https://issues.apache.org/jira/browse/ARROW-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774220#comment-16774220
]
Hatem Helal commented on ARROW-4650:
------------------------------------
I have a fix for this in this PR:
[https://github.com/apache/arrow/pull/3721]
Could you help with a review?
> The patch for PARQUET-1508 leads to infinite loop and infinite memory
> allocation when reading very sparse ByteArray columns
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: ARROW-4650
> URL: https://issues.apache.org/jira/browse/ARROW-4650
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Reporter: Valery Meleshkin
> Assignee: Hatem Helal
> Priority: Major
>
> In this loop
> [https://github.com/apache/arrow/commit/3d435e4f8d5fb7a54a4a9d285e1a42d60186d8dc#diff-47fe879cb9baad6c633c55f0a34a09c3R739]
> The branch of if dealing with null values does not increment variable 'i'.
> Therefore on chunks containing only NULLs once a thread enters the loop, it
> stays in that loop forever. I'm not entirely sure whether 'num_values'
> variable was meant to be the number of non-NULL values, yet the total number
> of values is passed here
> [https://github.com/apache/arrow/blob/3d435e4f8d5fb7a54a4a9d285e1a42d60186d8dc/cpp/src/parquet/arrow/record_reader.cc#L528]
>
> On my local machine adding `++i` to the NULL-handling branch seems to fix the
> problem. Unfortunately, I'm not familiar with the codebase enough to be
> certain it's a proper fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
