Re: DLQ Implementation

[XQ Hu via user]

You can check
https://github.com/search?q=repo%3Ajohnjcasey%2Fbeam%20withBadRecordErrorHandler&type=code.
The test codes show how to use them. More doc will be added later.

On Wed, Mar 27, 2024 at 7:15 PM Ruben Vargas 
wrote:

> Hello all
>
> Maybe a silly question. Are there any  suggestions for implementing a DLQ
> in my beam pipeline?
>
> Currently I'm using this library https://github.com/tosun-si/asgarde
> which is not bad, the only issue I found  is that sometimes it is hard  to
> use with GroupIntoBatches or other native transformations.
>
> Then I saw this PR https://github.com/apache/beam/pull/29164/files which
> I think is some sort of DLQ support? but I cannot find something on the
> docs for that and I am not 100% familiar with all the beam code to
> understand very well.
>
> Appreciated your help
>
>
> Thank you
> -Ruben
>

DLQ Implementation

[Ruben Vargas]

Hello all

Maybe a silly question. Are there any  suggestions for implementing a DLQ
in my beam pipeline?

Currently I'm using this library https://github.com/tosun-si/asgarde which
is not bad, the only issue I found  is that sometimes it is hard  to use
with GroupIntoBatches or other native transformations.

Then I saw this PR https://github.com/apache/beam/pull/29164/files which I
think is some sort of DLQ support? but I cannot find something on the docs
for that and I am not 100% familiar with all the beam code to understand
very well.

Appreciated your help


Thank you
-Ruben