Hi Pedro,
yes, I was more or less suggesting a similar approach to the one taken by
King. In code, it would look somewhat like this:
DataStream<T> input = ...;
DataStream<Tuple2<T, DynamicWindow>> withMyWindows = input.map(new
AssignMyWindows())
withMyWindows
.keyBy(...)
.window(new DynamicWindowAssigner())
...
where this is DynamicWindowAssigner:
public class DynamicWindowAssigner<T> extends WindowAssigner<Tuple2<T,
DynamicWindow>, TimeWindow> {
@Override
public Collection<TimeWindow> assignWindows(Tuple2<T,
DynamicWindow> element, long timestamp, WindowAssignerContext context) {
// extract some info from the dynamic window
return Collections.singletonList(new TimeWindow(..., ...)); // <-
this can also be a custom window
}
...
}
Cheers,
Aljoscha
On Thu, 3 Nov 2016 at 23:47 PedroMrChaves <pedro.mr.cha...@gmail.com> wrote:
> Hi,
>
> Thank you for the response.
>
> Can you give me an example?
> I'm new to Flink and I still don't understand all the constructs.
>
> I also read this article
> https://techblog.king.com/rbea-scalable-real-time-analytics-king/. They
> use a similar approach, but am still not understanding how assign windows.
>
> Regards,
> Pedro Chaves
>
>
>
> On Thu, Nov 3, 2016 at 6:02 PM, Aljoscha Krettek [via Apache Flink User
> Mailing List archive.] <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=9891&i=0>> wrote:
>
> Hi Pedro,
> you can have dynamic windows by assigning the windows to elements in your
> Processor (so you would need to extend that type to have a field for the
> window). Then, you can write a custom WindowAssigner that will simply get
> the window from an event and assign that as the internal window.
>
> Please let me know if you need more details.
>
> Cheers,
> Aljoscha
>
> On Thu, 3 Nov 2016 at 18:40 PedroMrChaves <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=9882&i=0>> wrote:
>
> Hello,
>
> Your tip was very helpful and I took a similar approach.
>
> I have something like this:
> class Processor extends RichCoFlatMapFunction<Event, Rule, String> {
> public void flatMap1(Event event, Collector<String> out) {
> process(event,out); // run the javscript (rules) against the
> incoming events
> }
>
> public void flatMap2(Rule rule , Collector<String> out) {
> // We add the rule to the list of existing rules
> addNewRule(rule)
> }
> }
>
> But know I face a new challenge, I don't have access to the windowed
> constructs of flink and I can't dynamically create new window aggregations
> inside the flatMap. At least not that I know of.
>
> Did you face a similar problem? Any Ideas?
>
> Thank you and regards,
> Pedro Chaves
>
>
>
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/What-is-the-best-way-to-load-add-patterns-dynamically-at-runtime-with-Flink-tp9461p9876.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/What-is-the-best-way-to-load-add-patterns-dynamically-at-runtime-with-Flink-tp9461p9882.html
> To unsubscribe from What is the best way to load/add patterns dynamically
> (at runtime) with Flink?, click here.
> NAML
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
>
> ------------------------------
> View this message in context: Re: What is the best way to load/add
> patterns dynamically (at runtime) with Flink?
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/What-is-the-best-way-to-load-add-patterns-dynamically-at-runtime-with-Flink-tp9461p9891.html>
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at
> Nabble.com.
>
