Hello all, When I run the pipeline with 4 samples (very small dataset), I don't get any error on DirectRunner or DataflowRunner
When I run it with 50 samples dataset, I get the following error for the run.wait_until_finished() What does this error mean? Thanks, Eila KeyErrorTraceback (most recent call last)<ipython-input-38-4d2dc0c0717f> in <module>() 1 result = pc.run()----> 2 result.wait_until_finish() /usr/local/envs/py2env/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.pyc in wait_until_finish(self, duration) 771 while thread.isAlive(): 772 time.sleep(5.0)--> 773 if self.state != PipelineState.DONE: 774 # TODO(BEAM-1290): Consider converting this to an error log based on the 775 # resolution of the issue. /usr/local/envs/py2env/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.pyc in state(self) 741 } 742 --> 743 return (api_jobstate_map[self._job.currentState] if self._job.currentState 744 else PipelineState.UNKNOWN) 745 KeyError: CurrentStateValueValuesEnum(JOB_STATE_PENDING, 9) -- Eila www.orielresearch.org https://www.meetup.com/Deep-Learning-In-Production/
