Have you tried to use option 2 with a virtual environment ? I think that should allow you to use python C extensions, although your cluster will need to have the same architecture and python versions...
On 17 December 2015 at 15:06, gzc <18810513...@126.com> wrote: > I write spouts and bolts mainly in python. I ran topology locally fine. > But when I want to deploy it to remote cluster, my python spouts and bolts > cannot run because remote cluster didn't have necessary dependencies. > > Now I have thought of two solutions to make it work and their respective cons > and pros. > > 1. Install necessary dependencies in machines where supervisor was deployed > This solution Can use python C extension. The disadvantage is that I need > `pip install -r requirements.txt` in all supervisors. And maybe version > confliction occur. > > 2. Put python dependencies into resource > Topology can run regardless of environment of supervisor. A .jar is an > adequate environment for python components. But cannot use python C > extensions. > Actually, I prefer this solution. But I haven't made decision. > > How do you manage python dependencies? What is the best practice? > > > > 冬天在被窝就可以处理邮件的APP，支持所有邮箱！马上使用享终身免费邮件提醒>> > <http://mail.163.com/dashi/?from=mail7> > -- Carlos Perelló Marínhttps://www.serverdensity.com
