I write spouts and bolts mainly in python. I ran topology locally fine. But when I want to deploy it to remote cluster, my python spouts and bolts cannot run because remote cluster didn't have necessary dependencies.
Now I have thought of two solutions to make it work and their respective cons and pros. 1. Install necessary dependencies in machines where supervisor was deployed This solution Can use python C extension. The disadvantage is that I need `pip install -r requirements.txt` in all supervisors. And maybe version confliction occur. 2. Put python dependencies into resource Topology can run regardless of environment of supervisor. A .jar is an adequate environment for python components. But cannot use python C extensions. Actually, I prefer this solution. But I haven't made decision. How do you manage python dependencies? What is the best practice?
