Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10233 )
Change subject: IMPALA-6946: handle negative counts in RLE decoder ...................................................................... IMPALA-6946: handle negative counts in RLE decoder This improves the handling of out-of-range values to avoid hitting various DCHECKs, including the one in the JIRA. repeat_count_ and literal_count_ are int32_ts. Avoid setting them to a negative value directly or by integer overflow. Switch to using uint32_t for "VLQ" encoding, which should be ULEB-128 encoding according to the Parquet standard. This fixes an infinite loop in PutVlqInt() for negative values - the bug was that shifting right sign-extended the signed value. Testing: Added backend test to exercise handling of large literal and repeat counts that don't fit in an int32_t. Before these fixes it could trigger several DCHECKs. Change-Id: If75ef3fb12494209918c100f26407cd93b17addb Reviewed-on: http://gerrit.cloudera.org:8080/10233 Reviewed-by: Lars Volker <l...@cloudera.com> Reviewed-by: Dan Hecht <dhe...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M be/src/util/bit-stream-utils.h M be/src/util/bit-stream-utils.inline.h M be/src/util/dict-encoding.h M be/src/util/rle-encoding.h M be/src/util/rle-test.cc 5 files changed, 90 insertions(+), 39 deletions(-) Approvals: Lars Volker: Looks good to me, but someone else must approve Dan Hecht: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10233 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If75ef3fb12494209918c100f26407cd93b17addb Gerrit-Change-Number: 10233 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Lars Volker <l...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
