Hi hagersaleh, You should know why the error occurred with large scale data. Broadcast variables can handle only data of which size is fit for single machine.
I meant that using an external system such as Redis, HBase, …, etc. The connection with the external system could be initialized in `open` method of rich functions such as `RichFilterFunction`, `RichFlatMapFunction`, …, etc.. You can choose another solution which Stephen said. He said that rethink your approach. I think that rethinking your algorithm would be better than my suggestion. From your code, I don’t understand why you want to use broadcast variable. You can do same thing with filter and join operations. Here is my implementation [1]. Regards, Chiwan Park [1] https://gist.github.com/chiwanpark/a0b0269c9a9b058d15d3 > On Sep 4, 2015, at 3:51 AM, hagersaleh <loveallah1...@yahoo.com> wrote: > > Hi Chiwan Park > not understand this solution please explain more > > > > -- > View this message in context: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/when-use-broadcast-variable-and-run-on-bigdata-display-this-error-please-help-tp2455p2676.html > Sent from the Apache Flink User Mailing List archive. mailing list archive at > Nabble.com.
