[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Impala Public Jenkins has posted comments on this change. Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/7870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Impala Public Jenkins has submitted this change and it was merged. Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. .. IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. Having the repetition level set to REPEATED on the root schema resulted a scan to fail with error when Impala tried to parse that table. As a solution, the 'REPEATED' repetition level is ignored when the root schema is processed. The reasoning behind is that the Parquet format description says that the repetition level of the root schema should not be set to REPEATED anyway, so it's safe to ignore it in case it is set to this value for some reason. Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 Reviewed-on: http://gerrit.cloudera.org:8080/7870 Reviewed-by: Dan Hecht Tested-by: Impala Public Jenkins --- M be/src/exec/parquet-metadata-utils.cc M testdata/data/README A testdata/data/repeated_root_schema.parquet M tests/query_test/test_scanners.py 4 files changed, 27 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Verified Dan Hecht: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/7870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Impala Public Jenkins has posted comments on this change. Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1199/ -- To view, visit http://gerrit.cloudera.org:8080/7870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Dan Hecht has posted comments on this change. Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/7870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Gabor Kaszab has posted comments on this change.
Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in
Parquet.
..
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7870/3/be/src/exec/parquet-metadata-utils.cc
File be/src/exec/parquet-metadata-utils.cc:
Line 368: !is_root_schema /*PARQUET-843*/) {
> I think it would be good to reference the parquet jira in a code comment he
Done
--
To view, visit http://gerrit.cloudera.org:8080/7870
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Gabor Kaszab
Gerrit-Reviewer: Alex Behm
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Dan Hecht
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Laszlo Gaal
Gerrit-Reviewer: Tim Armstrong
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Hello Lars Volker, Tim Armstrong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/7870 to look at the new patch set (#4). Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. .. IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. Having the repetition level set to REPEATED on the root schema resulted a scan to fail with error when Impala tried to parse that table. As a solution, the 'REPEATED' repetition level is ignored when the root schema is processed. The reasoning behind is that the Parquet format description says that the repetition level of the root schema should not be set to REPEATED anyway, so it's safe to ignore it in case it is set to this value for some reason. Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 --- M be/src/exec/parquet-metadata-utils.cc M testdata/data/README A testdata/data/repeated_root_schema.parquet M tests/query_test/test_scanners.py 4 files changed, 27 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/7870/4 -- To view, visit http://gerrit.cloudera.org:8080/7870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Lars Volker has posted comments on this change. Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. .. Patch Set 3: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/7870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Dan Hecht has posted comments on this change.
Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in
Parquet.
..
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7870/3/be/src/exec/parquet-metadata-utils.cc
File be/src/exec/parquet-metadata-utils.cc:
Line 368: !is_root_schema) {
I think it would be good to reference the parquet jira in a code comment here
since this code would probably be surprising given just the parquet spec.
--
To view, visit http://gerrit.cloudera.org:8080/7870
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Gabor Kaszab
Gerrit-Reviewer: Alex Behm
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Dan Hecht
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Laszlo Gaal
Gerrit-Reviewer: Tim Armstrong
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Tim Armstrong has posted comments on this change. Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. .. Patch Set 3: Any more comments Lars? -- To view, visit http://gerrit.cloudera.org:8080/7870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Tim Armstrong has posted comments on this change. Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. .. Patch Set 3: Code-Review+1 Thanks for running the tests and letting us know - the link isn't necessary since it's not useful to community members who don't work at Cloudera. -- To view, visit http://gerrit.cloudera.org:8080/7870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Gabor Kaszab has posted comments on this change. Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. .. Patch Set 3: Build and core tests are green: http://sandbox.jenkins.cloudera.com/job/impala-umbrella-build-and-test/12287/ -- To view, visit http://gerrit.cloudera.org:8080/7870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Gabor Kaszab has posted comments on this change. Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/7870/1//COMMIT_MSG Commit Message: Line 9: Having the repetition level set to REPEATED on the root schema > My reasoning was that it makes the commit message more self contained. When Done http://gerrit.cloudera.org:8080/#/c/7870/2//COMMIT_MSG Commit Message: PS2, Line 7: error during a sc > Here it says wrong result, but below it says that the queries failed. Can y Done -- To view, visit http://gerrit.cloudera.org:8080/7870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Tim Armstrong Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4826: Fix error during a scan on repeated root schema in Parquet.
Hello Tim Armstrong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/7870 to look at the new patch set (#3). Change subject: IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. .. IMPALA-4826: Fix error during a scan on repeated root schema in Parquet. Having the repetition level set to REPEATED on the root schema resulted a scan to fail with error when Impala tried to parse that table. As a solution, the 'REPEATED' repetition level is ignored when the root schema is processed. The reasoning behind is that the Parquet format description says that the repetition level of the root schema should not be set to REPEATED anyway, so it's safe to ignore it in case it is set to this value for some reason. Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 --- M be/src/exec/parquet-metadata-utils.cc M testdata/data/README A testdata/data/repeated_root_schema.parquet M tests/query_test/test_scanners.py 4 files changed, 27 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/7870/3 -- To view, visit http://gerrit.cloudera.org:8080/7870 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7ea84589e1d122ad9d43adde46893ec0ecc5f9c4 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Tim Armstrong
