Hi everyone!!

Working with multimap based side inputs on the global window I'm
experiencing something unexpected (at least to me) that I'd like to share
with you to clarify.

The way I understand multimaps is that when one emits two values for the
same key for the same window (obvious thing here as I'm working on the
Global one), the newly emitted values are appended to the Iterable
collection that is the value for that particular key on the map.

Testing it in this job (it is using scio, but side inputs are implemented
with PCollectionViews):
https://github.com/calonso/beam_experiments/blob/master/refreshingsideinput/src/main/scala/com/mrcalonso/RefreshingSideInput2.scala

The steps to reproduce are:
1. Create one table on the target BQ
2. Run the job
3. Patch the table on BQ (add one field), this should generate a new
TableSchema for the corresponding TableReference
4. An updated value of the fields number appear on the logs, but there is
only one element within the iterable, as if it had been updated instead of
appended!!

Is that the expected behaviour? Is a bug? Am I missing something?

Thanks!

Reply via email to