Hi,
i'm using Flink (v.1.11.2) and would like to use the StreamingFileSink for
writing into HDFS in Parquet format.
As it says in the documentation I have added the dependencies:
org.apache.flink
flink-parquet_${scala.binary.version}
${flink.version}
And this is my file sink definit
Hi Jan,
Could you have a try by adding this dependency ?
org.apache.parquet
parquet-avro
1.11.1
Best,
Yun
--Original Mail --
Sender:Jan Oelschlegel
Send Date:Thu Jan 14 00:49:30 2021
Recipients:user@flink.apache.org
Subject:StreamingFileSi
Hi Jan
Could you make sure you are packaging that dependency with your job jar?
There are instructions how to configure your build setup[1]. Especially
the part how to build a jar with dependencies might come in handy[2].
Best,
Dawid
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1
]
https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/project-configuration.html#create-project
Best,
Jan
Von: Dawid Wysakowicz
Gesendet: Donnerstag, 14. Januar 2021 12:42
An: Jan Oelschlegel ; user@flink.apache.org
Betreff: Re: StreamingFileSink with ParquetAvroWriters
Hi Jan
Could
:34
An: Jan Oelschlegel ; user@flink.apache.org
Betreff: Re: StreamingFileSink with ParquetAvroWriters
Hi Jan,
Could you have a try by adding this dependency ?
org.apache.parquet
parquet-avro
1.11.1
Best,
Yun
--Original Mail --
Sender:Jan
rg/projects/flink/flink-docs-release-1.11/dev/project-configuration.html#create-project
>
>
>
> Best,
>
> Jan
>
>
>
>
>
> *Von:*Dawid Wysakowicz
> *Gesendet:* Donnerstag, 14. Januar 2021 12:42
> *An:* Jan Oelschlegel ;
> user@flink.apache.org
>
Yes, after unzipping it is in the jar:
[cid:image002.jpg@01D6EB3D.33BCC530]
Von: Dawid Wysakowicz
Gesendet: Freitag, 15. Januar 2021 12:10
An: Jan Oelschlegel ; user@flink.apache.org
Betreff: Re: AW: StreamingFileSink with ParquetAvroWriters
Have you checked if the class (org/apache/parquet
Sorry, wrong build. It is not in the jar.
Von: Jan Oelschlegel
Gesendet: Freitag, 15. Januar 2021 12:52
An: Dawid Wysakowicz ; user@flink.apache.org
Betreff: AW: AW: StreamingFileSink with ParquetAvroWriters
Yes, after unzipping it is in the jar:
[cid:image001.jpg@01D6EB3F.136D8560]
Von
