As the message says, enable the api from cloud console of gcloud cli utility. (Refer dataflow docs)
If that’s enabled, ensure the robot account has the permissions it needs - dataflow api, gcs and anything else you are trying to access. On Thu, Nov 22, 2018 at 04:33 Unais T <tpun...@gmail.com> wrote: > I am trying to run a simple data flow job in google cloud - its running > perfectly in local - but when I tried to launch it I'm getting the > following error - I tried debug a lot > > can someone help on this > > INFO:root:Created job with id: [2018-11-22_02_57_07-12079060901530487381] > INFO:root:To access the Dataflow monitoring console, please navigate to > https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-22_02_57_07-12079060901530487381?project=kkfas-main-account-221808 > INFO:root:Job 2018-11-22_02_57_07-12079060901530487381 is in state > JOB_STATE_PENDING > INFO:root:2018-11-22T10:57:08.052Z: JOB_MESSAGE_DETAILED: Autoscaling is > enabled for job 2018-11-22_02_57_07-12079060901530487381. The number of > workers will be between 1 and 1000. > INFO:root:2018-11-22T10:57:08.072Z: JOB_MESSAGE_DETAILED: Autoscaling was > automatically enabled for job 2018-11-22_02_57_07-12079060901530487381. > INFO:root:2018-11-22T10:57:40.405Z: JOB_MESSAGE_ERROR: Workflow failed. > Causes: There was a problem refreshing your credentials. Please check: > 1. Dataflow API is enabled for your project. > 2. There is a robot service account for your project: > service-[project > number]@dataflow-service-producer-prod.iam.gserviceaccount.com should have > access to your project. If this account does not appear in the permissions > tab for your project, contact Dataflow support. > INFO:root:Job 2018-11-22_02_57_07-12079060901530487381 is in state > JOB_STATE_FAILED > Traceback (most recent call last): > File > "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", > line 162, in _run_module_as_main > "__main__", fname, loader, pkg_name) > File > "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", > line 72, in _run_code > exec code in run_globals > File "/Users/u/Projects//digital/test.py", line 49, in <module> > run() > File "/Users/u/Projects//dataflow//digital/test.py", line 44, in run > return p.run().wait_until_finish() > File > "/Users/u/VirtualEnv/dataflow/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", > line 1122, in wait_until_finish > (self.state, getattr(self._runner, 'last_error_msg', None)), self) > apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: > Dataflow pipeline failed. State: FAILED, Error: > Workflow failed. Causes: There was a problem refreshing your credentials. > Please check: > 1. Dataflow API is enabled for your project. > 2. There is a robot service account for your project: > service-[project > number]@dataflow-service-producer-prod.iam.gserviceaccount.com should have > access to your project. If this account does not appear in the permissions > tab for your project, contact Dataflow support. > >
