Can the Beam data model (specifically the Python SDK) support executing
functions that are idempotent but not concurrency-safe?

I am thinking of a task like setting up a database (or in my case, a Zarr
<https://zarr.dev/> store in Xarray-Beam
<https://github.com/google/xarray-beam>) where it is not safe to run setup
concurrently, but if the whole operation fails it is safe to retry.

I recognize that a better model would be to use entirely atomic operations,
but sometimes this can be challenging to guarantee for tools that were not
designed with parallel computing in mind.

Cheers,
Stephan

Reply via email to