That I believe is expected behavior. The windowing strategy is applied at the following group by key operation. To have the windows fire the way you want, try putting a group by key immediately after the desired windowing function.
The messages right now are being bundled aggressively for performance reasons and doing a gbk would ensure desired bundle delineations. Ankur Chauhan Sent from my iPhone > On May 6, 2017, at 14:11, Josh <[email protected]> wrote: > > Hi all, > I am using a PubSubIO source, windowing every 10 seconds and then doing > something with the windows, for example: > pipeline > .apply(PubsubIO.readPubsubMessagesWithAttributes().fromSubscription(..)) > .apply(Window.into(FixedWindows.of(Duration.standardSeconds(10)))) > .apply(MapElements > .into(TypeDescriptors.strings()) > .via((PubsubMessage msg) -> msg.getAttribute(..))) > .apply(new WriteOneFilePerWindow(..)); > My expectation was that if I publish a pubsub message, and then publish > another 10+ seconds later, a single file should be written for the previous > 10 second window. However I find that I need to publish a lot of messages for > any files to be written at all (e.g. 30+ messages). > Is this expected behaviour when using PubSubIO? Is there a way to tweak it to > fire the windows more eagerly? > Thanks, > Josh
