1. Yes, methods will only be called by one thread. The FLink API aims to
abstract all concurrency topics away when using the provided methods and
state.
2. The open() method should always be the first method being called. If
this is not the case, this is definitely a bug. Which Flink version are
you using? If it is 1.9, could you verify the behavior with 1.8? The
community recently simplified the architeture under the hood.
Thanks for your feedback.
Regards,
Timo
On 11.12.19 10:41, KristoffSC wrote:
Hi Vino,
Thank you for your response and provided links.
So just to clarify and small follow up.
1. Methods will be called only by one thread right?
2. The links you provided are tackling a case when we got a "fast stream"
element before we received broadcast stream element. In my case we had
Broadcast element first, before we got any "fast stream" element. Because
open method was not called (I've observed it will be called only before
first processElement method call, so before processing the first "fast
stream" element) we don't have the state descriptor which would be
initialized in open method. So we actually cannot "store/process" this
broadcast element in our broadcast state.
@Override
public void open(Configuration parameters) throws Exception {
super.open(parameters);
processingRulesDesc = new MapStateDescriptor<>(
"RulesBroadcastState",
Types.VOID,
TypeInformation.of(new TypeHint<ProcessingRule>() {
}));
}
In this case, bcState will be null since open method was not yet called.
public void processBroadcastElement(ProcessingRule rule, Context ctx,
Collector<EvaluatedTransaction> out) throws Exception {
// store the new pattern by updating the broadcast state
BroadcastState<Void, ProcessingRule> bcState =
ctx.getBroadcastState(processingRulesDesc);
bcState.put(null, rule);
}
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
