[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-02-17 Thread Code Review 
Gergely Fürnstáhl has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/18224


Change subject: IMPALA-10948: Default scale and DecimalType
..

IMPALA-10948: Default scale and DecimalType

Added default 0 for scale if it is not set to comply with parquet spec.

Wrapped reading scale and precision in a function to support reading
LogicalType.DecimalType if it is set, falling back to old ones if it is
not, for backward compatibility.

Regenerated bad_parquet_decimals table with filled DecimalType, moved
missing scale test, as it is no longer a bad table.

Added no_scale.parquet table to test reading table without set scale.

Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
---
M be/src/exec/parquet/parquet-data-converter.h
M be/src/exec/parquet/parquet-metadata-utils.cc
M testdata/bad_parquet_data/README
M testdata/bad_parquet_data/illegal_decimals.parq
M testdata/data/README
A testdata/data/no_scale.parquet
M 
testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test
M 
testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test
M tests/query_test/test_scanners.py
9 files changed, 117 insertions(+), 48 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/18224/2
--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 2
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-02-17 Thread Code Review 
Gergely Fürnstáhl has uploaded a new patch set (#3). ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..

IMPALA-10948: Default scale and DecimalType

Added default 0 for scale if it is not set to comply with parquet spec.

Wrapped reading scale and precision in a function to support reading
LogicalType.DecimalType if it is set, falling back to old ones if it is
not, for backward compatibility.

Regenerated bad_parquet_decimals table with filled DecimalType, moved
missing scale test, as it is no longer a bad table.

Added no_scale.parquet table to test reading table without set scale.

Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
---
M be/src/exec/parquet/parquet-data-converter.h
M be/src/exec/parquet/parquet-metadata-utils.cc
M testdata/bad_parquet_data/README
M testdata/bad_parquet_data/illegal_decimals.parq
M testdata/data/README
A testdata/data/no_scale.parquet
M 
testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test
M 
testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test
M tests/query_test/test_scanners.py
9 files changed, 120 insertions(+), 48 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/18224/3
--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 3
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-02-17 Thread Code Review 
Gergely Fürnstáhl has uploaded a new patch set (#4). ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..

IMPALA-10948: Default scale and DecimalType

Added default 0 for scale if it is not set to comply with parquet spec.

Wrapped reading scale and precision in a function to support reading
LogicalType.DecimalType if it is set, falling back to old ones if it is
not, for backward compatibility.

Regenerated bad_parquet_decimals table with filled DecimalType, moved
missing scale test, as it is no longer a bad table.

Added no_scale.parquet table to test reading table without set scale.

Checked it with parquet-tools:
message schema {
  optional fixed_len_byte_array(2) d1 (DECIMAL(4,0));
}

Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
---
M be/src/exec/parquet/parquet-data-converter.h
M be/src/exec/parquet/parquet-metadata-utils.cc
M testdata/bad_parquet_data/README
M testdata/bad_parquet_data/illegal_decimals.parq
M testdata/data/README
A testdata/data/no_scale.parquet
M 
testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test
M 
testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test
M tests/query_test/test_scanners.py
9 files changed, 120 insertions(+), 48 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/18224/4
--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 4
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-02-17 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/10169/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 2
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 17 Feb 2022 10:49:11 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-02-17 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 3:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/10170/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 3
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 17 Feb 2022 10:52:14 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-02-17 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 4:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/10171/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 4
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 17 Feb 2022 10:59:47 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-02-17 Thread Zoltan Borok-Nagy (Code Review) 
Zoltan Borok-Nagy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 4:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-data-converter.h
File be/src/exec/parquet/parquet-data-converter.h:

http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-data-converter.h@74
PS4, Line 74: if (parquet_element_->__isset.logicalType
: && parquet_element_->logicalType.__isset.DECIMAL)
:   return parquet_element_->logicalType.DECIMAL.precision;
nit: multi-line if stmts should use braces.


http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-metadata-utils.cc
File be/src/exec/parquet/parquet-metadata-utils.cc:

http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-metadata-utils.cc@208
PS4, Line 208: Precision is required, this should be called after checking 
IsPrecisionSet
We could add a DCHECK(IsPrecisionSet(schema_element));


http://gerrit.cloudera.org:8080/#/c/18224/4/testdata/data/README
File testdata/data/README:

http://gerrit.cloudera.org:8080/#/c/18224/4/testdata/data/README@682
PS4, Line 682: .__set_scale(1);
Do we need this line?


http://gerrit.cloudera.org:8080/#/c/18224/4/testdata/data/README@684
PS4, Line 684: +  file_metadata_.schema[1].logicalType.DECIMAL.scale = 1;
 : +  file_metadata_.schema[1].logicalType.__isset.DECIMAL = false;
Are these lines needed?


http://gerrit.cloudera.org:8080/#/c/18224/4/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:

http://gerrit.cloudera.org:8080/#/c/18224/4/tests/query_test/test_scanners.py@393
PS4, Line 393:
nit: unnecessary blank line


http://gerrit.cloudera.org:8080/#/c/18224/4/tests/query_test/test_scanners.py@394
PS4, Line 394: default-scale
default-scale.test is not added to the PS.



--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 4
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 17 Feb 2022 11:11:34 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-02-17 Thread Code Review 
Gergely Fürnstáhl has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 4:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-data-converter.h
File be/src/exec/parquet/parquet-data-converter.h:

http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-data-converter.h@74
PS4, Line 74: if (parquet_element_->__isset.logicalType
: && parquet_element_->logicalType.__isset.DECIMAL)
:   return parquet_element_->logicalType.DECIMAL.precision;
> nit: multi-line if stmts should use braces.
Done


http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-metadata-utils.cc
File be/src/exec/parquet/parquet-metadata-utils.cc:

http://gerrit.cloudera.org:8080/#/c/18224/4/be/src/exec/parquet/parquet-metadata-utils.cc@208
PS4, Line 208: Precision is required, this should be called after checking 
IsPrecisionSet
> We could add a DCHECK(IsPrecisionSet(schema_element));
Done


http://gerrit.cloudera.org:8080/#/c/18224/4/testdata/data/README
File testdata/data/README:

http://gerrit.cloudera.org:8080/#/c/18224/4/testdata/data/README@682
PS4, Line 682: .__set_scale(1);
> Do we need this line?
With scale=1 I wanted to showcase it does not read out the default 0 from 
there, rather getting it from the getter.


http://gerrit.cloudera.org:8080/#/c/18224/4/testdata/data/README@684
PS4, Line 684: +  file_metadata_.schema[1].logicalType.DECIMAL.scale = 1;
 : +  file_metadata_.schema[1].logicalType.__isset.DECIMAL = false;
> Are these lines needed?
With scale=1 I wanted to showcase it does not read out the default 0 from 
there, rather getting it from the getter.

Possibly__isset.logicalType =false is enough, but I think the intention is more 
clear this way


http://gerrit.cloudera.org:8080/#/c/18224/4/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:

http://gerrit.cloudera.org:8080/#/c/18224/4/tests/query_test/test_scanners.py@393
PS4, Line 393:
> nit: unnecessary blank line
Done


http://gerrit.cloudera.org:8080/#/c/18224/4/tests/query_test/test_scanners.py@394
PS4, Line 394: default-scale
> default-scale.test is not added to the PS.
Done



--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 4
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 17 Feb 2022 12:08:28 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-02-17 Thread Code Review 
Gergely Fürnstáhl has uploaded a new patch set (#5). ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..

IMPALA-10948: Default scale and DecimalType

Added default 0 for scale if it is not set to comply with parquet spec.

Wrapped reading scale and precision in a function to support reading
LogicalType.DecimalType if it is set, falling back to old ones if it is
not, for backward compatibility.

Regenerated bad_parquet_decimals table with filled DecimalType, moved
missing scale test, as it is no longer a bad table.

Added no_scale.parquet table to test reading table without set scale.

Checked it with parquet-tools:
message schema {
  optional fixed_len_byte_array(2) d1 (DECIMAL(4,0));
}

Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
---
M be/src/exec/parquet/parquet-data-converter.h
M be/src/exec/parquet/parquet-metadata-utils.cc
M testdata/bad_parquet_data/README
M testdata/bad_parquet_data/illegal_decimals.parq
M testdata/data/README
A testdata/data/no_scale.parquet
A testdata/workloads/functional-query/queries/QueryTest/default-scale.test
M 
testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test
M 
testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test
M tests/query_test/test_scanners.py
10 files changed, 126 insertions(+), 48 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/18224/5
--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 5
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-02-17 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 5:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/10173/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 5
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 17 Feb 2022 12:33:41 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-03-01 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7891/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 5
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Tue, 01 Mar 2022 17:55:46 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-03-01 Thread Zoltan Borok-Nagy (Code Review) 
Zoltan Borok-Nagy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 5: Code-Review+2

Looks great! Thanks for fixing this issue.


--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 5
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Tue, 01 Mar 2022 17:54:37 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-03-01 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 5: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 5
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Tue, 01 Mar 2022 22:40:46 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-03-03 Thread Code Review 
Gergely Fürnstáhl has uploaded a new patch set (#6). ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..

IMPALA-10948: Default scale and DecimalType

Added default 0 for scale if it is not set to comply with parquet spec.

Wrapped reading scale and precision in a function to support reading
LogicalType.DecimalType if it is set, falling back to old ones if it is
not, for backward compatibility.

Regenerated bad_parquet_decimals table with filled DecimalType, moved
missing scale test, as it is no longer a bad table.

Added no_scale.parquet table to test reading table without set scale.

Checked it with parquet-tools:
message schema {
  optional fixed_len_byte_array(2) d1 (DECIMAL(4,0));
}

Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
---
M be/src/exec/parquet/parquet-data-converter.h
M be/src/exec/parquet/parquet-metadata-utils.cc
M testdata/bad_parquet_data/README
M testdata/bad_parquet_data/illegal_decimals.parq
M testdata/data/README
A testdata/data/no_scale.parquet
A testdata/workloads/functional-query/queries/QueryTest/default-scale.test
M 
testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test
M 
testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test
M tests/query_test/test_scanners.py
10 files changed, 126 insertions(+), 48 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/18224/6
--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 6
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-03-03 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 6:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/10246/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 6
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Thu, 03 Mar 2022 17:49:57 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-03-04 Thread Zoltan Borok-Nagy (Code Review) 
Zoltan Borok-Nagy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 6: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 6
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 04 Mar 2022 11:57:15 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-03-04 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7902/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 6
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 04 Mar 2022 11:57:34 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-03-04 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..

IMPALA-10948: Default scale and DecimalType

Added default 0 for scale if it is not set to comply with parquet spec.

Wrapped reading scale and precision in a function to support reading
LogicalType.DecimalType if it is set, falling back to old ones if it is
not, for backward compatibility.

Regenerated bad_parquet_decimals table with filled DecimalType, moved
missing scale test, as it is no longer a bad table.

Added no_scale.parquet table to test reading table without set scale.

Checked it with parquet-tools:
message schema {
  optional fixed_len_byte_array(2) d1 (DECIMAL(4,0));
}

Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Reviewed-on: http://gerrit.cloudera.org:8080/18224
Reviewed-by: Zoltan Borok-Nagy 
Tested-by: Impala Public Jenkins 
---
M be/src/exec/parquet/parquet-data-converter.h
M be/src/exec/parquet/parquet-metadata-utils.cc
M testdata/bad_parquet_data/README
M testdata/bad_parquet_data/illegal_decimals.parq
M testdata/data/README
A testdata/data/no_scale.parquet
A testdata/workloads/functional-query/queries/QueryTest/default-scale.test
M 
testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test
M 
testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test
M tests/query_test/test_scanners.py
10 files changed, 126 insertions(+), 48 deletions(-)

Approvals:
  Zoltan Borok-Nagy: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 7
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy

[Impala-ASF-CR] IMPALA-10948: Default scale and DecimalType

2022-03-04 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/18224 )

Change subject: IMPALA-10948: Default scale and DecimalType
..


Patch Set 6: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/18224
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I003220b6e2ef39d25d1c33df62c8432803fdc6eb
Gerrit-Change-Number: 18224
Gerrit-PatchSet: 6
Gerrit-Owner: Gergely Fürnstáhl 
Gerrit-Reviewer: Gergely Fürnstáhl 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Zoltan Borok-Nagy 
Gerrit-Comment-Date: Fri, 04 Mar 2022 16:49:21 +
Gerrit-HasComments: No