[
https://issues.apache.org/jira/browse/BEAM-11006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pablo Estrada updated BEAM-11006:
---------------------------------
Status: Resolved (was: Triage Needed)
> Allow Failsafe Handling of BigQuery Streaming Writes
> ----------------------------------------------------
>
> Key: BEAM-11006
> URL: https://issues.apache.org/jira/browse/BEAM-11006
> Project: Beam
> Issue Type: Improvement
> Components: extensions-java-gcp
> Reporter: Dylan Hercher
> Priority: P2
> Labels: Clarified, bigquery, google-cloud-bigquery
> Original Estimate: 336h
> Time Spent: 2.5h
> Remaining Estimate: 333.5h
>
> To allow handling of a generic failsafe (of any type) would allow a dead
> letter queue to retain the original source data rather than the cleaned
> version and could be more easily understood and re-processed.
>
> The BigQueryIO.Write currently supports `withFormatFunction` which allows for
> a serializable function to be applied to each datapoint -> TableRow. Ideally
> that same source value could be converted with a separate function:
> `withFailsafeFormatFunction` taken (InputT -> TableRow) or possibly (InputT
> -> OutputT), though the backwards compatibility of OutputT is more difficult.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
