Thanks Martijn, are there any alternatives to write to GCS using Flink 1.14? Recoverability isn't important for my current use case.
On Mon, Mar 14, 2022 at 10:58 AM Martijn Visser <martijnvis...@apache.org> wrote: > Hi Bobby, > > That's because Flink 1.14 currently doesn't support the RecoverableWriter > for GCS. This will be supported as of Flink 1.15, which you can see in the > relevant Flink Jira ticket [1]. For more details on why RecoverableWriter > is important, the JavaDoc [2] is probably the best source of information. > > Best regards, > > Martijn Visser > https://twitter.com/MartijnVisser82 > > [1] https://issues.apache.org/jira/browse/FLINK-11838 > [2] > https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/core/fs/RecoverableWriter.html > > > On Mon, 14 Mar 2022 at 15:21, Bobby Richard <bobby.rich...@broadcom.com> > wrote: > >> I am receiving the following exception when attempting to write to GCS >> with the FileSink in FLink 14.3. Using flink hadoop shaded 2.8.3-10.0 >> and gcs connector hadoop2-2.1.1. >> >> >> java.lang.UnsupportedOperationException: Recoverable writers on Hadoop >> are only supported for HDFS >> >> I am able to write checkpoints and savepoints to GCS successfully. >> >> This electronic communication and the information and any files >> transmitted with it, or attached to it, are confidential and are intended >> solely for the use of the individual or entity to whom it is addressed and >> may contain information that is confidential, legally privileged, protected >> by privacy laws, or otherwise restricted from disclosure to anyone else. If >> you are not the intended recipient or the person responsible for delivering >> the e-mail to the intended recipient, you are hereby notified that any use, >> copying, distributing, dissemination, forwarding, printing, or copying of >> this e-mail is strictly prohibited. If you received this e-mail in error, >> please return the e-mail to the sender, delete it from your computer, and >> destroy any printed copy of it. > > -- This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it.
smime.p7s
Description: S/MIME Cryptographic Signature
