My problem I generated a service account - it has all permission and I downloaded the credentials.json for that - But when I'm trying to launch the job - I think it still taking the default account.
Is some configuration I have to do take the service account I created? On Thu, Nov 22, 2018 at 10:08 PM Ankur Chauhan <[email protected]> wrote: > As the message says, enable the api from cloud console of gcloud cli > utility. (Refer dataflow docs) > > If that’s enabled, ensure the robot account has the permissions it needs - > dataflow api, gcs and anything else you are trying to access. > > On Thu, Nov 22, 2018 at 04:33 Unais T <[email protected]> wrote: > >> I am trying to run a simple data flow job in google cloud - its running >> perfectly in local - but when I tried to launch it I'm getting the >> following error - I tried debug a lot >> >> can someone help on this >> >> INFO:root:Created job with id: [2018-11-22_02_57_07-12079060901530487381] >> INFO:root:To access the Dataflow monitoring console, please navigate to >> https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-22_02_57_07-12079060901530487381?project=kkfas-main-account-221808 >> INFO:root:Job 2018-11-22_02_57_07-12079060901530487381 is in state >> JOB_STATE_PENDING >> INFO:root:2018-11-22T10:57:08.052Z: JOB_MESSAGE_DETAILED: Autoscaling is >> enabled for job 2018-11-22_02_57_07-12079060901530487381. The number of >> workers will be between 1 and 1000. >> INFO:root:2018-11-22T10:57:08.072Z: JOB_MESSAGE_DETAILED: Autoscaling was >> automatically enabled for job 2018-11-22_02_57_07-12079060901530487381. >> INFO:root:2018-11-22T10:57:40.405Z: JOB_MESSAGE_ERROR: Workflow failed. >> Causes: There was a problem refreshing your credentials. Please check: >> 1. Dataflow API is enabled for your project. >> 2. There is a robot service account for your project: >> service-[project >> number]@dataflow-service-producer-prod.iam.gserviceaccount.com should have >> access to your project. If this account does not appear in the permissions >> tab for your project, contact Dataflow support. >> INFO:root:Job 2018-11-22_02_57_07-12079060901530487381 is in state >> JOB_STATE_FAILED >> Traceback (most recent call last): >> File >> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", >> line 162, in _run_module_as_main >> "__main__", fname, loader, pkg_name) >> File >> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", >> line 72, in _run_code >> exec code in run_globals >> File "/Users/u/Projects//digital/test.py", line 49, in <module> >> run() >> File "/Users/u/Projects//dataflow//digital/test.py", line 44, in run >> return p.run().wait_until_finish() >> File >> "/Users/u/VirtualEnv/dataflow/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", >> line 1122, in wait_until_finish >> (self.state, getattr(self._runner, 'last_error_msg', None)), self) >> apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: >> Dataflow pipeline failed. State: FAILED, Error: >> Workflow failed. Causes: There was a problem refreshing your credentials. >> Please check: >> 1. Dataflow API is enabled for your project. >> 2. There is a robot service account for your project: >> service-[project >> number]@dataflow-service-producer-prod.iam.gserviceaccount.com should have >> access to your project. If this account does not appear in the permissions >> tab for your project, contact Dataflow support. >> >>
