Re: About Flink parquet format

2023-09-24 Thread Feng Jin
ly without closing? > > > > Rgds, > > Kamal > > > > *From:* Kamal Mittal via user > *Sent:* 21 September 2023 07:58 AM > *To:* Feng Jin > *Cc:* user@flink.apache.org > *Subject:* RE: About Flink parquet format > > > > Yes. > > > > Du

RE: About Flink parquet format

2023-09-24 Thread Kamal Mittal via user
Hello, Can you please share that why Flink is not able to handle exception and keeps on creating files continuously without closing? Rgds, Kamal From: Kamal Mittal via user Sent: 21 September 2023 07:58 AM To: Feng Jin Cc: user@flink.apache.org Subject: RE: About Flink parquet format Yes

RE: About Flink parquet format

2023-09-20 Thread Kamal Mittal via user
k.apache.org> Subject: Re: About Flink parquet format Hi Kamal What exception did you encounter? I have tested it locally and it works fine. Best, Feng On Mon, Sep 18, 2023 at 11:04 AM Kamal Mittal mailto:kamal.mit...@ericsson.com>> wrote: Hello, Checkpointing is enabled and works f

Re: About Flink parquet format

2023-09-20 Thread Feng Jin
> This is coming from class CapacityByteArrayOutputStream and contained in > parquet-common library. > > > > Rgds, > > Kamal > > > > *From:* Feng Jin > *Sent:* 19 September 2023 01:01 PM > *To:* Kamal Mittal > *Cc:* user@flink.apache.org > *Subject:* Re: About Fli

RE: About Flink parquet format

2023-09-19 Thread Kamal Mittal via user
To: Kamal Mittal Cc: user@flink.apache.org Subject: Re: About Flink parquet format Hi Kamal What exception did you encounter? I have tested it locally and it works fine. Best, Feng On Mon, Sep 18, 2023 at 11:04 AM Kamal Mittal mailto:kamal.mit...@ericsson.com>> wrote: Hello, Checkpo

Re: About Flink parquet format

2023-09-19 Thread Feng Jin
bytes as otherwise there is exception thrown at back-end. > > > > Looks to be an issue which is not handled by file sink bulk writer? > > > > Rgds, > > Kamal > > > > *From:* Feng Jin > *Sent:* 15 September 2023 04:14 PM > *To:* Kamal Mittal > *Cc:* user@flink.ap

RE: About Flink parquet format

2023-09-17 Thread Kamal Mittal via user
Mittal Cc: user@flink.apache.org Subject: Re: About Flink parquet format Hi Kamal Check if the checkpoint of the task is enabled and triggered correctly. By default, write parquet files will roll a new file when checkpointing. Best, Feng On Thu, Sep 14, 2023 at 7:27 PM Kamal Mittal via user

Re: About Flink parquet format

2023-09-15 Thread Feng Jin
Hi Kamal Check if the checkpoint of the task is enabled and triggered correctly. By default, write parquet files will roll a new file when checkpointing. Best, Feng On Thu, Sep 14, 2023 at 7:27 PM Kamal Mittal via user wrote: > Hello, > > > > Tried parquet file creation with file sink bulk