答复: 答复: how to generate a column using mapParition and then add it back to the df?

Hi guha, Thanks a lot! This is perfectly what I want and I'll try to implement it. MoTao 发件人: ayan guha 发送时间: 2016年8月8日 18:05:37 收件人: 莫涛 抄送: ndj...@gmail.com; user@spark.apache.org 主题: Re: 答复: how to generate a column using mapParition and

Re: 答复: how to generate a column using mapParition and then add it back to the df?

Hi I think you should modify initModel() function to getOrCreateModel() and create the model as singleton object. You may want to refer this link On Mon, Aug 8, 2016 at 7:44 PM, 莫涛

答复: how to generate a column using mapParition and then add it back to the df?

Hi Ndjido, Thanks for your reply. Yes, it is good idea if the model can be broadcast. I'm working with a built library (on Linux, say classifier.so and classifier.h) and it requires the model file is in the local file system. As I don't have access to the library code, I write JNI to wrap the