Hi all,
I am using a PubSubIO source, windowing every 10 seconds and then doing
something with the windows, for example:

pipeline
    .apply(PubsubIO.readPubsubMessagesWithAttributes().fromSubscription(..))
    .apply(Window.into(FixedWindows.of(Duration.standardSeconds(10))))
    .apply(MapElements
        .into(TypeDescriptors.strings())
        .via((PubsubMessage msg) -> msg.getAttribute(*..*)))
    .apply(new WriteOneFilePerWindow(..));

My expectation was that if I publish a pubsub message, and then
publish another 10+ seconds later, a single file should be written for
the previous 10 second window. However I find that I need to publish a
lot of messages for any files to be written at all (e.g. 30+
messages).

Is this expected behaviour when using PubSubIO? Is there a way to
tweak it to fire the windows more eagerly?

Thanks,

Josh

Reply via email to