Thank you very much for the fast reply, Hans. Issue was that I looked not at Parquet native Types but in general available types like [1]. In addition to that I find it strange that Big Number gets mapped to String. But I will not create ticket for that now, that I saw this was done somewhat intentionally.
Kind Regards Jonathan https://docs.microsoft.com/en-us/common-data-model/sdk/parquet-to-cdm-datatype Von: Hans Van Akelyen <[email protected]> Gesendet: Freitag, 5. August 2022 14:49 An: [email protected] Betreff: Re: PARQUET FILE OUTPUT Column Datatypes Hi Jonathan, We translate all our types to the supported Parquet native types which are documented here [1]. We have not implemented the notion of the logical types in this output transform. The cast from Hop types to Parquet types can be found here [2]. All of our data types should be supported and changing the type using the metadata tab from select values should work. If you feel this is not the case feel free to create a jira ticket, preferably with a reproduction pipeline attached. Kind regards, Hans [1] https://parquet.apache.org/docs/file-format/types/ [2] https://github.com/apache/hop/blob/master/plugins/tech/parquet/src/main/java/org/apache/hop/parquet/transforms/output/ParquetOutput.java#L166 On Fri, 5 Aug 2022 at 09:30, Jonathan Haehnel <[email protected]<mailto:[email protected]>> wrote: Dear Hop Team, I have a question regarding the PARQUET FILE OUTPUT Step in Hop (2.0). I would like to define the column Data Type for each column that comes from some Database in the Parquet File. I figured defining Datatypes by the Select Value Step should do the trick. However it seems no matter what I choose the metadata for each column to be, it only seems to save columns with following datatypes. INT64, timestamp, string and double. I would like to save a column as decimal, date and smaller int types. Is that at all possible with this step? Kind Regards Jonathan Jonathan Haehnel Business Intelligence Engineer Travian Games GmbH | Wilhelm-Wagenfeld-Str. 22 | 80807 Munich | Germany Tel.: +49 / (0)89 / 324 915 144 [email protected]<mailto:[email protected]> http://www.traviangames.com<http://www.traviangames.com/> [cid:[email protected]] Registered office: Munich Commercial register number: HRB 173511 Managing Directors: Jörg Strathaus, Frederik Hammes VAT ID: DE246258085 This email and its attachments are strictly confidential and are intended solely for the attention of the person to whom it is addressed. If you are not the intended recipient of this email, please delete it including its attachments immediately and inform us accordingly.
