I'm using version 2.51.0 and The configuration is like this
write
.withoutValidation()
.withCreateDisposition(BigQueryIO.Write.CreateDisposition.CREATE_NEVER)
.withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_APPEND)
.withExtendedErrorInfo()
Currently error handling is implemented on sinks in an ad-hoc basis
(if at all) but John (cc'd) is looking at improving things here.
On Mon, Dec 4, 2023 at 10:25 AM Juan Romero wrote:
>
> Hi guys. I want to ask you about how to deal with the scenario when the
> target sink (eg: jdbc, kafka,
Hi,
Could you add some more detail? Which beam version are you using?
On Tue, Dec 5, 2023 at 1:52 PM hsy...@gmail.com wrote:
> Any one has experience in writing to BQ using storage api
>
> I tried to use it because according to the document it is more efficient
> but I got error below
>
>
Any one has experience in writing to BQ using storage api
I tried to use it because according to the document it is more efficient
but I got error below
2023-12-05 04:01:29.741 PST
Error message from worker: java.lang.RuntimeException:
java.lang.IllegalStateException
I guess it's about not documented --flink_submit_uber_jar pipeline option.
Best regards,
Stanislav Porotikov
From: Alexey Romanenko
Sent: Tuesday, December 5, 2023 3:48 PM
To: user
Subject: Re: Why do we need Job Server?
Oh, interesting. I didn’t know about that possibility, thanks
—
Alexey
Oh, interesting. I didn’t know about that possibility, thanks
—
Alexey
> On 4 Dec 2023, at 18:14, Robert Bradshaw via user
> wrote:
>
> Note that this shouldn't be strictly necessary, e.g. for Python one can embed
> the pipeline definition into the jar itself which is then just uploaded as
