[
https://issues.apache.org/jira/browse/IMPALA-8381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16827373#comment-16827373
]
ASF subversion and git services commented on IMPALA-8381:
---------------------------------------------------------
Commit 6a703741d8fdc359833a0d593ca8b121cd5d890d in impala's branch
refs/heads/master from Daniel Becker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=6a70374 ]
IMPALA-8381: Optimize ParquetPlainEncoder::DecodeBatch() for simple types
Refactored the ParquetPlainEncoder::Decode() and
ParquetPlainEncoder::DecodeBatch() methods to increase performance in
batch decoding.
The `Decode` and `DecodeBatch` methods retain their behaviour and
outward interface, but the internal structure changes.
We change how we split up the `Decode` template specialisations. The
generic unspecialised template is used for numerical parquet types
(INT32, INT64, INT96, FLOAT and DOUBLE) and various specialisations are
used for BYTE_ARRAY and FIXED_LEN_BYTE_ARRAY.
We add a new method template, DecodeNoCheck, which does the actual
decoding without bounds checking. It is called by the generic Decode
method template internally. For all parquet types except for BYTE_ARRAY,
DecodeBatch performs the bounds check once for the whole batch at the
same time and calls DecodeNoCheck, so we save the cost of bounds
checking for every decoded value. For BYTE_ARRAY, this cannot be done
and we have to perform the checks for every value.
In the non-BYTE_ARRAY version of DecodeBatch, we explicitly unroll the
loop in batches of 8 to increase performance.
The overall performance increase is up to 2x for small strides (8 bytes,
INT32) but decreases as the stride increases, and disappears from around
40 bytes. With bigger strides, there is no performance difference from
the previous implementation.
Testing:
Added tests to parquet-plain-test.cc to test the `Decode` and the
`DecodeBatch` methods both in single-value decoding and batch
decoding.
Change-Id: I57b7d2573bb6dfd038e581acb3bd8ea1565aa20d
Reviewed-on: http://gerrit.cloudera.org:8080/12985
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Remove branch from ParquetPlainEncoder::Decode()
> ------------------------------------------------
>
> Key: IMPALA-8381
> URL: https://issues.apache.org/jira/browse/IMPALA-8381
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Csaba Ringhofer
> Assignee: Daniel Becker
> Priority: Minor
> Labels: newbie, parquet, performance, ramp-up
>
> Removing the "if" at
> https://github.com/apache/impala/blob/5670f96b828d57f9e36510bb9af02bcc31de775c/be/src/exec/parquet/parquet-common.h#L203
> can lead to 1.5x speed up in plain decoding (type=int32, stride=16). For
> primitive types, the same check can be done for a whole batch, so the speedup
> can be gained for large batches without loosing safety. The only Parquet type
> where this check is needed per element is BYTE_ARRAY (typically used for
> STRING columns), which already has a template specialization for
> ParquetPlainEncoder::Decode().
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
