[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Gergely Fürnstáhl has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18224 Change subject: IMPALA-10948: Default scale and DecimalType .. IMPALA-10948: Default scale and DecimalType Added default 0 for scale if it is not set to comply with parquet spec. Wrapped reading scale and precision in a function to support reading LogicalType.DecimalType if it is set, falling back to old ones if it is not, for backward compatibility. Regenerated bad_parquet_decimals table with filled DecimalType, moved missing scale test, as it is no longer a bad table. Added no_scale.parquet table to test reading table without set scale. Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb --- M be/src/exec/parquet/parquet-data-converter.h M be/src/exec/parquet/parquet-metadata-utils.cc M testdata/bad_parquet_data/README M testdata/bad_parquet_data/illegal_decimals.parq M testdata/data/README A testdata/data/no_scale.parquet M testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test M testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test M tests/query_test/test_scanners.py 9 files changed, 117 insertions(+), 48 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/18224/2 -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 2 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Gergely Fürnstáhl has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. IMPALA-10948: Default scale and DecimalType Added default 0 for scale if it is not set to comply with parquet spec. Wrapped reading scale and precision in a function to support reading LogicalType.DecimalType if it is set, falling back to old ones if it is not, for backward compatibility. Regenerated bad_parquet_decimals table with filled DecimalType, moved missing scale test, as it is no longer a bad table. Added no_scale.parquet table to test reading table without set scale. Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb --- M be/src/exec/parquet/parquet-data-converter.h M be/src/exec/parquet/parquet-metadata-utils.cc M testdata/bad_parquet_data/README M testdata/bad_parquet_data/illegal_decimals.parq M testdata/data/README A testdata/data/no_scale.parquet M testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test M testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test M tests/query_test/test_scanners.py 9 files changed, 120 insertions(+), 48 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/18224/3 -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 3 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Gergely Fürnstáhl has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. IMPALA-10948: Default scale and DecimalType Added default 0 for scale if it is not set to comply with parquet spec. Wrapped reading scale and precision in a function to support reading LogicalType.DecimalType if it is set, falling back to old ones if it is not, for backward compatibility. Regenerated bad_parquet_decimals table with filled DecimalType, moved missing scale test, as it is no longer a bad table. Added no_scale.parquet table to test reading table without set scale. Checked it with parquet-tools: message schema { optional fixed_len_byte_array(2) d1 (DECIMAL(4,0)); } Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb --- M be/src/exec/parquet/parquet-data-converter.h M be/src/exec/parquet/parquet-metadata-utils.cc M testdata/bad_parquet_data/README M testdata/bad_parquet_data/illegal_decimals.parq M testdata/data/README A testdata/data/no_scale.parquet M testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test M testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test M tests/query_test/test_scanners.py 9 files changed, 120 insertions(+), 48 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/18224/4 -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 4 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10169/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 2 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 17 Feb 2022 10:49:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10170/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 3 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 17 Feb 2022 10:52:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10171/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 4 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 17 Feb 2022 10:59:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 4: (6 comments) http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-data-converter.h File be/src/exec/parquet/parquet-data-converter.h: http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-data-converter.h@74 PS4, Line 74: if (parquet_element_->__isset.logicalType : && parquet_element_->logicalType.__isset.DECIMAL) : return parquet_element_->logicalType.DECIMAL.precision; nit: multi-line if stmts should use braces. http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-metadata-utils.cc File be/src/exec/parquet/parquet-metadata-utils.cc: http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-metadata-utils.cc@208 PS4, Line 208: Precision is required, this should be called after checking IsPrecisionSet We could add a DCHECK(IsPrecisionSet(schema_element)); http://gerrit.cloudera.org:8080/#/c/18224/4/testdata/data/README File testdata/data/README: http://gerrit.cloudera.org:8080/#/c/18224/4/testdata/data/README@682 PS4, Line 682: .__set_scale(1); Do we need this line? http://gerrit.cloudera.org:8080/#/c/18224/4/testdata/data/README@684 PS4, Line 684: + file_metadata_.schema[1].logicalType.DECIMAL.scale = 1; : + file_metadata_.schema[1].logicalType.__isset.DECIMAL = false; Are these lines needed? http://gerrit.cloudera.org:8080/#/c/18224/4/tests/query_test/test_scanners.py File tests/query_test/test_scanners.py: http://gerrit.cloudera.org:8080/#/c/18224/4/tests/query_test/test_scanners.py@393 PS4, Line 393: nit: unnecessary blank line http://gerrit.cloudera.org:8080/#/c/18224/4/tests/query_test/test_scanners.py@394 PS4, Line 394: default-scale default-scale.test is not added to the PS. -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 4 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 17 Feb 2022 11:11:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Gergely Fürnstáhl has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 4: (6 comments) http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-data-converter.h File be/src/exec/parquet/parquet-data-converter.h: http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-data-converter.h@74 PS4, Line 74: if (parquet_element_->__isset.logicalType : && parquet_element_->logicalType.__isset.DECIMAL) : return parquet_element_->logicalType.DECIMAL.precision; > nit: multi-line if stmts should use braces. Done http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-metadata-utils.cc File be/src/exec/parquet/parquet-metadata-utils.cc: http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-metadata-utils.cc@208 PS4, Line 208: Precision is required, this should be called after checking IsPrecisionSet > We could add a DCHECK(IsPrecisionSet(schema_element)); Done http://gerrit.cloudera.org:8080/#/c/18224/4/testdata/data/README File testdata/data/README: http://gerrit.cloudera.org:8080/#/c/18224/4/testdata/data/README@682 PS4, Line 682: .__set_scale(1); > Do we need this line? With scale=1 I wanted to showcase it does not read out the default 0 from there, rather getting it from the getter. http://gerrit.cloudera.org:8080/#/c/18224/4/testdata/data/README@684 PS4, Line 684: + file_metadata_.schema[1].logicalType.DECIMAL.scale = 1; : + file_metadata_.schema[1].logicalType.__isset.DECIMAL = false; > Are these lines needed? With scale=1 I wanted to showcase it does not read out the default 0 from there, rather getting it from the getter. Possibly__isset.logicalType =false is enough, but I think the intention is more clear this way http://gerrit.cloudera.org:8080/#/c/18224/4/tests/query_test/test_scanners.py File tests/query_test/test_scanners.py: http://gerrit.cloudera.org:8080/#/c/18224/4/tests/query_test/test_scanners.py@393 PS4, Line 393: > nit: unnecessary blank line Done http://gerrit.cloudera.org:8080/#/c/18224/4/tests/query_test/test_scanners.py@394 PS4, Line 394: default-scale > default-scale.test is not added to the PS. Done -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 4 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 17 Feb 2022 12:08:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Gergely Fürnstáhl has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. IMPALA-10948: Default scale and DecimalType Added default 0 for scale if it is not set to comply with parquet spec. Wrapped reading scale and precision in a function to support reading LogicalType.DecimalType if it is set, falling back to old ones if it is not, for backward compatibility. Regenerated bad_parquet_decimals table with filled DecimalType, moved missing scale test, as it is no longer a bad table. Added no_scale.parquet table to test reading table without set scale. Checked it with parquet-tools: message schema { optional fixed_len_byte_array(2) d1 (DECIMAL(4,0)); } Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb --- M be/src/exec/parquet/parquet-data-converter.h M be/src/exec/parquet/parquet-metadata-utils.cc M testdata/bad_parquet_data/README M testdata/bad_parquet_data/illegal_decimals.parq M testdata/data/README A testdata/data/no_scale.parquet A testdata/workloads/functional-query/queries/QueryTest/default-scale.test M testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test M testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test M tests/query_test/test_scanners.py 10 files changed, 126 insertions(+), 48 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/18224/5 -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 5 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10173/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 5 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 17 Feb 2022 12:33:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7891/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 5 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 01 Mar 2022 17:55:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 5: Code-Review+2 Looks great! Thanks for fixing this issue. -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 5 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 01 Mar 2022 17:54:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 5 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 01 Mar 2022 22:40:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Gergely Fürnstáhl has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. IMPALA-10948: Default scale and DecimalType Added default 0 for scale if it is not set to comply with parquet spec. Wrapped reading scale and precision in a function to support reading LogicalType.DecimalType if it is set, falling back to old ones if it is not, for backward compatibility. Regenerated bad_parquet_decimals table with filled DecimalType, moved missing scale test, as it is no longer a bad table. Added no_scale.parquet table to test reading table without set scale. Checked it with parquet-tools: message schema { optional fixed_len_byte_array(2) d1 (DECIMAL(4,0)); } Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb --- M be/src/exec/parquet/parquet-data-converter.h M be/src/exec/parquet/parquet-metadata-utils.cc M testdata/bad_parquet_data/README M testdata/bad_parquet_data/illegal_decimals.parq M testdata/data/README A testdata/data/no_scale.parquet A testdata/workloads/functional-query/queries/QueryTest/default-scale.test M testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test M testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test M tests/query_test/test_scanners.py 10 files changed, 126 insertions(+), 48 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/18224/6 -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 6 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/10246/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 6 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 03 Mar 2022 17:49:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 6 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 04 Mar 2022 11:57:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7902/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 6 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 04 Mar 2022 11:57:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. IMPALA-10948: Default scale and DecimalType Added default 0 for scale if it is not set to comply with parquet spec. Wrapped reading scale and precision in a function to support reading LogicalType.DecimalType if it is set, falling back to old ones if it is not, for backward compatibility. Regenerated bad_parquet_decimals table with filled DecimalType, moved missing scale test, as it is no longer a bad table. Added no_scale.parquet table to test reading table without set scale. Checked it with parquet-tools: message schema { optional fixed_len_byte_array(2) d1 (DECIMAL(4,0)); } Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Reviewed-on: http://gerrit.cloudera.org:8080/18224 Reviewed-by: Zoltan Borok-Nagy Tested-by: Impala Public Jenkins --- M be/src/exec/parquet/parquet-data-converter.h M be/src/exec/parquet/parquet-metadata-utils.cc M testdata/bad_parquet_data/README M testdata/bad_parquet_data/illegal_decimals.parq M testdata/data/README A testdata/data/no_scale.parquet A testdata/workloads/functional-query/queries/QueryTest/default-scale.test M testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test M testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test M tests/query_test/test_scanners.py 10 files changed, 126 insertions(+), 48 deletions(-) Approvals: Zoltan Borok-Nagy: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 7 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18224 ) Change subject: IMPALA-10948: Default scale and DecimalType .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/18224 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb Gerrit-Change-Number: 18224 Gerrit-PatchSet: 6 Gerrit-Owner: Gergely Fürnstáhl Gerrit-Reviewer: Gergely Fürnstáhl Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 04 Mar 2022 16:49:21 + Gerrit-HasComments: No