Hello Everybody, I’m currently refactoring some code and am looking for a better alternative to handle JPMML-Models in data streams. At the moment the flink job I’m working on references a model-object as a Singleton which I want to change because static references tend to cause problems in distributed systems.
I thought about handing the model-object to the function that uses it as a variable closure. The object can be between 16MB and 250MB in size (depending on the depth of the decision tree). According to https://cwiki.apache.org/confluence/display/FLINK/Variables+Closures+vs.+Broadcast+Variables that’s way too large though. Are there any viable alternatives or would this be the „right way“ to handle this situation? Best Regards, Julian ************************************************************************************************************** bonprix Handelsgesellschaft mbH Sitz der Gesellschaft: Hamburg Geschäftsführung: Dr. Marcus Ackermann (Vorsitzender) Dr. Kai Heck Rien Jansen Markus Fuchshofen Beiratsvorsitzender: Alexander Birken Handelsregister AG Hamburg HR B 36 455 Adresse: bonprix Handelsgesellschaft mbH Haldesdorfer Str. 61 22179 Hamburg Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet. This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received the e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. **************************************************************************************************************
