mattcasters opened a new pull request, #8684: URL: https://github.com/apache/hop/pull/8684
Parquet Output mapped a Hop Date to a Parquet TIMESTAMP(MILLIS) column. BigQuery then imported that column as a timestamp, and the fields grid had no way to choose DATE or any other logical type. Each output field can now set a Parquet type, plus precision and scale for Decimal. Leaving the type empty keeps the previous mapping, so existing pipelines still write Date as TIMESTAMP(MILLIS). Get Fields proposes Date for a Hop date, TimestampMicros for a timestamp, and Decimal when a BigNumber fits in a decimal. An explicit type replaces the column after the automatic schema conversion and is the type written to the file. Invalid precision, scale, Int32 range, and unknown type names fail before a file is written. `./mvnw -pl plugins/tech/parquet test -Dtest=ParquetLogicalTypeTest,ParquetFieldTest,ParquetOutputMetaTest,ParquetWriteSupportTest,ParquetOutputTest,HopToParquetTypeMatrixTest -Dsurefire.failIfNoSpecifiedTests=false` passed (86 tests). `./mvnw -pl plugins/tech/parquet test -Dsurefire.failIfNoSpecifiedTests=false` passed (511 tests). `./mvnw -pl plugins/tech/parquet spotless:apply` made no further edits. The full `mvn clean install apache-rat:check` was not run, and Hop GUI was not started. fixes #3864 ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Run `mvn clean install apache-rat:check` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If you have a group of commits related to the same change, please squash your commits into one and force push your branch using `git rebase -i`. - [x] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. To make clear that you license your contribution under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) you have to acknowledge this by using the following check-box. - [x] I hereby declare this contribution to be licensed under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) - [ ] In any other case, please file an [Apache Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
