@Philipp, I didn't notice that earlier. Thank you for adding that to 'streampipes-pipeline-elements-all' module. :)
@Patrick, +1 to the idea of adding the window concept to the sdk/core. As you mentioned since aggregations and even sequences/patterns can leverage window concepts, it'll be really useful to have it at that level. I'll go through the sdk/core and see how window concept can be generify to be used at that level. Grainier Perera. On Wed, 3 Jun 2020 at 20:45, Patrick Wiener <[email protected]> wrote: > Hi Grainier, > > indeed a very nice extension to the toolbox. > > What I was thinking of is the following. > > Since you basically implemented window-semantic in plain java this is > something we could need for various pipeline elements, e.g. a lightweight > aggregation component in Java other than in Flink. > > So maybe it is a good idea to add such capabilities straight to the > SDK/engine > so that in case it is needed, pipeline element developers can leverage it. > > What do you guys think? > > Patrick > > > Am 03.06.2020 um 14:14 schrieb Philipp Zehnder <[email protected]>: > > > > Hi Grainier, > > > > very cool component, especially the cron window is a very good idea. > > I have to admit, I have to get used to the syntax ;) but the feature is > super useful and this is a good way to make this component very flexible. > > > > By the way, in case you didn’t see it so far. In the extension project > there is a module 'streampipes-pipeline-elements-all' and there I also > registered your new components. > > We use this project for the StreamPipes ‘lite’ version. All single host > pipeline elements are registered there. Then we just have to start a single > container and reduce the memory footprint for StreamPipes. > > > > Philipp > > > > > >> On 1. Jun 2020, at 14:59, Grainier Perera <[email protected]> wrote: > >> > >> Thanks Dominik. I've also added Cron window support with RateLimiting. > So > >> it gives the flexibility of having various time triggers instead of > fixed > >> intervals. What do you think? > >> > >> Issue: https://issues.apache.org/jira/browse/STREAMPIPES-151 > >> PR: https://github.com/apache/incubator-streampipes-extensions/pull/20 > >> > >> Grainier Perera. > >> > >> > >> On Mon, 1 Jun 2020 at 03:11, Dominik Riemer <[email protected]> wrote: > >> > >>> This is super cool! > >>> That's a component we should definitely highlight in the next release > as > >>> it can be used together with many other pipeline elements. > >>> > >>> Btw, I also moved the pipeline element overview from the documentation > to > >>> the main website, so that it's easier for users to discover available > >>> pipeline elements and made the list searchable: > >>> https://streampipes.apache.org/pipeline-elements.html > >>> > >>> Can't wait to add the new elements to this list after the next release > 😊 > >>> > >>> Dominik > >>> > >>> -----Original Message----- > >>> From: Grainier Perera <[email protected]> > >>> Sent: Sunday, May 31, 2020 4:48 PM > >>> To: [email protected] > >>> Subject: PE to rate-limit events > >>> > >>> Hi all, > >>> > >>> As per the discussion on [1], I've implemented a PE to rate-limit > events. > >>> For now, it supports the following configurations, and I believe it > covers > >>> most rate-limit use cases. What do you think? > >>> > >>> - window type: time/length > >>> - window size. > >>> - event selection: first/last/all > >>> - event grouping: by field (for each unique field value, it'll create > a > >>> separate window) > >>> > >>> Issue: https://issues.apache.org/jira/browse/STREAMPIPES-147 > >>> PR: https://github.com/apache/incubator-streampipes-extensions/pull/18 > >>> [1] > >>> > >>> > https://lists.apache.org/thread.html/r836afda725bab338d81174c3f37b425f3cdd95c7b8699b1841d93170%40%3Cdev.streampipes.apache.org%3E > >>> > >>> Grainier Perera. > >>> > >>> > > > > > >
