The application is deployed successfully and can be scheduled/ launched via cron. it creates the dataflow job successfully and then the job fails.
The flow reads data from Bigquery, filters rows and writes to Bigquery. The filter step fails (beam.Filter(....)) *error details from google error report:* ImportError: No module named dataflow_pipeline at _import_module (/usr/local/lib/python2.7/dist-packages/dill/dill.py:767) at load_reduce (/usr/lib/python2.7/pickle.py:1133) at load (/usr/lib/python2.7/pickle.py:858) at load (/usr/local/lib/python2.7/dist-packages/dill/dill.py:266) at loads (/usr/local/lib/python2.7/dist-packages/dill/dill.py:277) at loads (/usr/local/lib/python2.7/dist-packages/apache_beam/internal/pickler.py:225) at dataflow_worker.operations.DoOperation.start (dataflow_worker/operations.c:9775) (operations.py:289) at dataflow_worker.operations.DoOperation.start (dataflow_worker/operations.c:10574) (operations.py:284) at dataflow_worker.operations.DoOperation.start (dataflow_worker/operations.c:10680) (operations.py:283) at execute (/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py:166) at do_work (/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py:581) when running dataflow manually from client, works fine! any suggestions and comments would be really helpful! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/b4c8f43d-4b98-48e8-bd8a-5e090af8c717%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
