Pooja Nilangekar has uploaded a new change for review. http://gerrit.cloudera.org:8080/7513
Change subject: Build a ConcatenatedStreams wrapper for ScannerContext::Stream ...................................................................... Build a ConcatenatedStreams wrapper for ScannerContext::Stream The ConcatenatedStreams class keeps track of a vector of streams which can be utilized by the BaseScalarColumnReader to read transparently from a list of streams. The ConcatenatedStreams would hide the different ScanRanges from the caller and provide the abstraction of a single stream. The class uses a list of streams and a index to point to the current active stream. The class moves to the next stream once the current stream reaches its end of stream. TODO: Currently, the ConcatenatedStreams don't support reads which span across multiple streams. Thus the maximum bytes you can read at any given instance is dictated by the number of bytes remaining in the current active stream. Testing: The existing test_scanners.py test was used to validate the correctness of the ConcatenatedStreams. Change-Id: I026f7a005ba4d6fc412ca0884f4fa0459da9c885 --- M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/parquet-column-readers.h M be/src/exec/scanner-context.cc M be/src/exec/scanner-context.h 4 files changed, 125 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/13/7513/1 -- To view, visit http://gerrit.cloudera.org:8080/7513 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I026f7a005ba4d6fc412ca0884f4fa0459da9c885 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Pooja Nilangekar <pooja.nilange...@cloudera.com> Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
