[Impala-ASF-CR] IMPALA-7360: sequence scanners sometimes skip blocks
Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11062 ) Change subject: IMPALA-7360: sequence scanners sometimes skip blocks .. IMPALA-7360: sequence scanners sometimes skip blocks The handling of sync markers after processing a block was broken - eos_ was set if the sync marker straddles the boundary. The expected behaviour (documented by comments) in this case is that the current scanner should process the next block, if there is one. If you look at the logic before the IMPALA-3905 change in commit 931bf49cd90e496df6bf260ae668ec6944f0016c, it split the checking of eosr() and eof() similar to this patch. Testing: Add regression tests that scans a large table with a variety of different scan range lengths, with some randomisation to exercise different edge cases. This reliably triggered the bug. Change-Id: I49a70a4925b0271204b8eea4f980299d7654805a Reviewed-on: http://gerrit.cloudera.org:8080/11062 Reviewed-by: Michael Ho Tested-by: Impala Public Jenkins --- M be/src/exec/base-sequence-scanner.cc A testdata/workloads/tpch/queries/tpch-scan-range-lengths.test M tests/query_test/test_scanners.py 3 files changed, 86 insertions(+), 20 deletions(-) Approvals: Michael Ho: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/11062 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I49a70a4925b0271204b8eea4f980299d7654805a Gerrit-Change-Number: 11062 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Pranay Singh Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7360: sequence scanners sometimes skip blocks
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11062 ) Change subject: IMPALA-7360: sequence scanners sometimes skip blocks .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11062 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I49a70a4925b0271204b8eea4f980299d7654805a Gerrit-Change-Number: 11062 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Pranay Singh Gerrit-Comment-Date: Fri, 27 Jul 2018 04:47:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7360: sequence scanners sometimes skip blocks
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11062 ) Change subject: IMPALA-7360: sequence scanners sometimes skip blocks .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2868/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/11062 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I49a70a4925b0271204b8eea4f980299d7654805a Gerrit-Change-Number: 11062 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Pranay Singh Gerrit-Comment-Date: Fri, 27 Jul 2018 01:32:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7360: sequence scanners sometimes skip blocks
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/11062 ) Change subject: IMPALA-7360: sequence scanners sometimes skip blocks .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11062 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I49a70a4925b0271204b8eea4f980299d7654805a Gerrit-Change-Number: 11062 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Pranay Singh Gerrit-Comment-Date: Fri, 27 Jul 2018 01:32:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7360: sequence scanners sometimes skip blocks
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11062 ) Change subject: IMPALA-7360: sequence scanners sometimes skip blocks .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/79/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11062 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I49a70a4925b0271204b8eea4f980299d7654805a Gerrit-Change-Number: 11062 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pranay Singh Gerrit-Comment-Date: Thu, 26 Jul 2018 22:12:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7360: sequence scanners sometimes skip blocks
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11062 ) Change subject: IMPALA-7360: sequence scanners sometimes skip blocks .. Patch Set 3: Build Started https://jenkins.impala.io/job/gerrit-code-review-checks/79/ Running initial code review checks. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11062 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I49a70a4925b0271204b8eea4f980299d7654805a Gerrit-Change-Number: 11062 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pranay Singh Gerrit-Comment-Date: Thu, 26 Jul 2018 21:33:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7360: sequence scanners sometimes skip blocks
Hello Pranay Singh, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11062 to look at the new patch set (#3). Change subject: IMPALA-7360: sequence scanners sometimes skip blocks .. IMPALA-7360: sequence scanners sometimes skip blocks The handling of sync markers after processing a block was broken - eos_ was set if the sync marker straddles the boundary. The expected behaviour (documented by comments) in this case is that the current scanner should process the next block, if there is one. If you look at the logic before the IMPALA-3905 change in commit 931bf49cd90e496df6bf260ae668ec6944f0016c, it split the checking of eosr() and eof() similar to this patch. Testing: Add regression tests that scans a large table with a variety of different scan range lengths, with some randomisation to exercise different edge cases. This reliably triggered the bug. Change-Id: I49a70a4925b0271204b8eea4f980299d7654805a --- M be/src/exec/base-sequence-scanner.cc A testdata/workloads/tpch/queries/tpch-scan-range-lengths.test M tests/query_test/test_scanners.py 3 files changed, 86 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/11062/3 -- To view, visit http://gerrit.cloudera.org:8080/11062 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I49a70a4925b0271204b8eea4f980299d7654805a Gerrit-Change-Number: 11062 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pranay Singh
[Impala-ASF-CR] IMPALA-7360: sequence scanners sometimes skip blocks
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11062 ) Change subject: IMPALA-7360: sequence scanners sometimes skip blocks .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-code-review-checks/78/ Running initial code review checks. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11062 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I49a70a4925b0271204b8eea4f980299d7654805a Gerrit-Change-Number: 11062 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pranay Singh Gerrit-Comment-Date: Thu, 26 Jul 2018 21:30:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7360: sequence scanners sometimes skip blocks
Hello Pranay Singh, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11062 to look at the new patch set (#2). Change subject: IMPALA-7360: sequence scanners sometimes skip blocks .. IMPALA-7360: sequence scanners sometimes skip blocks The handling of sync markers after processing a block was broken - eos_ was set if the sync marker straddles the boundary. The expected behaviour (documented by comments) in this case is that the current scanner should process the next block, if there is one. If you look at the logic before the IMPALA-3905 change in commit 931bf49cd90e496df6bf260ae668ec6944f0016c, it split the checking of eosr() and eof() similar to this patch. Testing: Add regression tests that scans a large table with a variety of different scan range lengths, with some randomisation to exercise different edge cases. This reliably triggered the bug. Change-Id: I49a70a4925b0271204b8eea4f980299d7654805a --- M be/src/exec/base-sequence-scanner.cc A testdata/workloads/tpch/queries/tpch-scan-range-lengths.test M tests/query_test/test_scanners.py 3 files changed, 86 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/11062/2 -- To view, visit http://gerrit.cloudera.org:8080/11062 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I49a70a4925b0271204b8eea4f980299d7654805a Gerrit-Change-Number: 11062 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Pranay Singh
