RE: Beam detached mode?

2023-12-04 Thread Поротиков Станислав Вячеславович via user
Thank you! But can we achive it throw the Beam, by providing some settings to Python pipeline? From: Pavel Solomin Sent: Monday, December 4, 2023 6:27 PM To: user@beam.apache.org Cc: Поротиков Станислав Вячеславович Subject: Re: Beam detached mode? Hello! Yes. You can run it like this:

Streaming management exception in the sink target.

2023-12-04 Thread Juan Romero
Hi guys. I want to ask you about how to deal with the scenario when the target sink (eg: jdbc, kafka, bigquery, pubsub etc) fails for any reason and i don't want to lost the message and create a bottleneck with many errors due an hypothetical target sink problem, and i want to use

Re: Beam detached mode?

2023-12-04 Thread Pavel Solomin
Hello! Yes. You can run it like this: flink run --class com.my.BeamFlinkApp --detached ... Best Regards, Pavel Solomin Tel: +351 962 950 692 | Skype: pavel_solomin | Linkedin On Mon, 4 Dec 2023 at 12:45, Поротиков Станислав Вячеславович via user

Beam detached mode?

2023-12-04 Thread Поротиков Станислав Вячеславович via user
Hello! I there any option in beam to run pipeline in detached mode? I want to submit job to Flink and exit from pipeline without any errors. Best regards, Stanislav Porotikov

Re: [Question] Does SnowflakeIO connector support for AWS and Azure?

2023-12-04 Thread Sachin Mittal
I think along with the storage integration name you also have to specify the snowpipe to use. SnowflakeIO.<>write() .withDataSourceConfiguration(dc) .to(TABLE_NAME) .withStagingBucketName(stagingBucketName)