Hi Bruno, Of course you can do that as well. (That's the good part :p )
I will open a PR soon with the proposed changes (first without breaking the current Api) and I will post it here. Cheers, Gyula On Tuesday, April 21, 2015, Bruno Cadonna <cado...@informatik.hu-berlin.de> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi Gyula, > > I have a question regarding your suggestion. > > Can the current continuous aggregation be also specified with your > proposed periodic aggregation? > > I am thinking about something like > > dataStream.reduce(...).every(Count.of(1)) > > Cheers, > Bruno > > On 20.04.2015 22:32, Gyula Fóra wrote: > > Hey all, > > > > I think we are missing a quite useful feature that could be > > implemented (with some slight modifications) on top of the current > > windowing api. > > > > We currently provide 2 ways of aggregating (or reducing) over > > streams: doing a continuous aggregation and always output the > > aggregated value (which cannot be done properly in parallel) or > > doing aggregation in a window periodically. > > > > What we don't have at the moment is periodic aggregations on the > > whole stream. I would even go as far as to remove the continuous > > outputting reduce/aggregate it and replace it with this version as > > this in return can be done properly in parallel. > > > > My suggestion would be that a call: > > > > dataStream.reduce(..) dataStream.sum(..) > > > > would return a windowed data stream where the window is the whole > > record history, and the user would need to define a trigger to get > > the actual reduced values like: > > > > dataStream.reduce(...).every(Time.of(4,sec)) to get the actual > > reduced results. dataStream.sum(...).every(...) > > > > I think the current data stream reduce/aggregation is very > > confusing without being practical for any normal use-case. > > > > Also this would be a very api breaking change (but I would still > > make this change as it is much more intuitive than the current > > behaviour) so I would try to push it before the release if we can > > agree. > > > > Cheers, Gyula > > > > - -- > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Dr. Bruno Cadonna > Postdoctoral Researcher > > Databases and Information Systems > Department of Computer Science > Humboldt-Universität zu Berlin > > http://www.informatik.hu-berlin.de/~cadonnab > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > > iQEcBAEBAgAGBQJVNgaeAAoJEKdCIJx7flKwfiMH/AzPpKtse9eMOzFsXSuBslNr > PZRQ0vpI7vw9eYFIuqp33SltN0zmLmDt3VzgJz0EZK5zSRCF9NOeke1emQwlrPsB > g65a4XccWT2qPotodF39jTTdE5epeUf8NdE552sr+Ya5LMtt8TmozD0lEOVfNt7n > R6KQdDU70U0zoCPwv0S13cak8a8k7phGvShXeW4nSZKp8C+WJa3IbUZkHlIlkC1L > OnyYy4b14bnfjiknKt2mKcjLG7eQEq0X6aN85Zf+5X8BUg3auk9N9Cva2XMRuD1p > gOoC+2gPZcr2IB9Sgs+s5pxfhaoVpbQ9Z7gRh8BkWqftveA7RD6KymmBxoUtujA= > =8bVQ > -----END PGP SIGNATURE----- >
