-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Gyula,
I have a question regarding your suggestion. Can the current continuous aggregation be also specified with your proposed periodic aggregation? I am thinking about something like dataStream.reduce(...).every(Count.of(1)) Cheers, Bruno On 20.04.2015 22:32, Gyula Fóra wrote: > Hey all, > > I think we are missing a quite useful feature that could be > implemented (with some slight modifications) on top of the current > windowing api. > > We currently provide 2 ways of aggregating (or reducing) over > streams: doing a continuous aggregation and always output the > aggregated value (which cannot be done properly in parallel) or > doing aggregation in a window periodically. > > What we don't have at the moment is periodic aggregations on the > whole stream. I would even go as far as to remove the continuous > outputting reduce/aggregate it and replace it with this version as > this in return can be done properly in parallel. > > My suggestion would be that a call: > > dataStream.reduce(..) dataStream.sum(..) > > would return a windowed data stream where the window is the whole > record history, and the user would need to define a trigger to get > the actual reduced values like: > > dataStream.reduce(...).every(Time.of(4,sec)) to get the actual > reduced results. dataStream.sum(...).every(...) > > I think the current data stream reduce/aggregation is very > confusing without being practical for any normal use-case. > > Also this would be a very api breaking change (but I would still > make this change as it is much more intuitive than the current > behaviour) so I would try to push it before the release if we can > agree. > > Cheers, Gyula > - -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dr. Bruno Cadonna Postdoctoral Researcher Databases and Information Systems Department of Computer Science Humboldt-Universität zu Berlin http://www.informatik.hu-berlin.de/~cadonnab ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEcBAEBAgAGBQJVNgaeAAoJEKdCIJx7flKwfiMH/AzPpKtse9eMOzFsXSuBslNr PZRQ0vpI7vw9eYFIuqp33SltN0zmLmDt3VzgJz0EZK5zSRCF9NOeke1emQwlrPsB g65a4XccWT2qPotodF39jTTdE5epeUf8NdE552sr+Ya5LMtt8TmozD0lEOVfNt7n R6KQdDU70U0zoCPwv0S13cak8a8k7phGvShXeW4nSZKp8C+WJa3IbUZkHlIlkC1L OnyYy4b14bnfjiknKt2mKcjLG7eQEq0X6aN85Zf+5X8BUg3auk9N9Cva2XMRuD1p gOoC+2gPZcr2IB9Sgs+s5pxfhaoVpbQ9Z7gRh8BkWqftveA7RD6KymmBxoUtujA= =8bVQ -----END PGP SIGNATURE-----
