See
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/4475/display/redirect?page=changes>
Changes:
[tgroh] Enable Proto Round Trips in the DirectRunner Tests
[tgroh] Add a suborder parameter to Networks#topologicalOrder
[tgroh] Add QueryablePipeline#forTransforms
------------------------------------------
[...truncated 1.13 MB...]
root: INFO: 2018-03-21T17:00:45.513Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2018-03-21T17:00:45.539Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2018-03-21T17:00:45.566Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2018-03-21T17:00:45.599Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2018-03-21T17:00:45.632Z: JOB_MESSAGE_DETAILED: Fusing consumer
pair_with_one into split
root: INFO: 2018-03-21T17:00:45.658Z: JOB_MESSAGE_DETAILED: Fusing consumer
group/Reify into pair_with_one
root: INFO: 2018-03-21T17:00:45.680Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/GroupByKey/Reify into
write/Write/WriteImpl/WindowInto(WindowIntoFn)
root: INFO: 2018-03-21T17:00:45.715Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WindowInto(WindowIntoFn) into write/Write/WriteImpl/Pair
root: INFO: 2018-03-21T17:00:45.738Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/WriteBundles into format
root: INFO: 2018-03-21T17:00:45.754Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/Pair into write/Write/WriteImpl/WriteBundles/WriteBundles
root: INFO: 2018-03-21T17:00:45.783Z: JOB_MESSAGE_DETAILED: Fusing consumer
split into read/Read
root: INFO: 2018-03-21T17:00:45.808Z: JOB_MESSAGE_DETAILED: Fusing consumer
count into group/GroupByWindow
root: INFO: 2018-03-21T17:00:45.842Z: JOB_MESSAGE_DETAILED: Fusing consumer
format into count
root: INFO: 2018-03-21T17:00:45.876Z: JOB_MESSAGE_DETAILED: Fusing consumer
group/Write into group/Reify
root: INFO: 2018-03-21T17:00:45.894Z: JOB_MESSAGE_DETAILED: Fusing consumer
group/GroupByWindow into group/Read
root: INFO: 2018-03-21T17:00:45.914Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/Extract into
write/Write/WriteImpl/GroupByKey/GroupByWindow
root: INFO: 2018-03-21T17:00:45.947Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/GroupByKey/Write into
write/Write/WriteImpl/GroupByKey/Reify
root: INFO: 2018-03-21T17:00:46.004Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/GroupByKey/GroupByWindow into
write/Write/WriteImpl/GroupByKey/Read
root: INFO: 2018-03-21T17:00:46.029Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2018-03-21T17:00:46.056Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2018-03-21T17:00:46.081Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2018-03-21T17:00:46.124Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2018-03-21T17:00:46.157Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2018-03-21T17:00:46.206Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2018-03-21T17:00:46.250Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/InitializeWrite into write/Write/WriteImpl/DoOnce/Read
root: INFO: 2018-03-21T17:00:46.293Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey2 into
write/Write/WriteImpl/PreFinalize/PreFinalize
root: INFO: 2018-03-21T17:00:46.321Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey2 into
write/Write/WriteImpl/PreFinalize/PreFinalize
root: INFO: 2018-03-21T17:00:46.345Z: JOB_MESSAGE_DEBUG: Workflow config is
missing a default resource spec.
root: INFO: 2018-03-21T17:00:46.382Z: JOB_MESSAGE_DEBUG: Adding StepResource
setup and teardown to workflow graph.
root: INFO: 2018-03-21T17:00:46.419Z: JOB_MESSAGE_DEBUG: Adding workflow start
and stop steps.
root: INFO: 2018-03-21T17:00:46.444Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
root: INFO: 2018-03-21T17:00:46.594Z: JOB_MESSAGE_DEBUG: Executing wait step
start26
root: INFO: 2018-03-21T17:00:46.661Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/DoOnce/Read+write/Write/WriteImpl/InitializeWrite+write/Write/WriteImpl/PreFinalize/MapToVoidKey0+write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0+write/Write/WriteImpl/WriteBundles/MapToVoidKey0+write/Write/WriteImpl/PreFinalize/MapToVoidKey0+write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0+write/Write/WriteImpl/WriteBundles/MapToVoidKey0
root: INFO: 2018-03-21T17:00:46.681Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/GroupByKey/Create
root: INFO: 2018-03-21T17:00:46.692Z: JOB_MESSAGE_DEBUG: Starting worker pool
setup.
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-f...
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: Job 2018-03-21_10_00_41-9840485252195089948 is in state
JOB_STATE_RUNNING
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-f...
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-f...
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-f...
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-f...
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-f...
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-f...
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-f...
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-f...
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-f...
root: INFO: 2018-03-21T17:00:46.711Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: 2018-03-21T17:00:56.366Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised
the number of workers to 0 based on the rate of progress in the currently
running step(s).
root: INFO: 2018-03-21T17:00:56.451Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/GroupByKey/Session" materialized.
root: INFO: 2018-03-21T17:00:56.482Z: JOB_MESSAGE_DEBUG: Value "group/Session"
materialized.
root: INFO: 2018-03-21T17:00:56.539Z: JOB_MESSAGE_BASIC: Executing operation
read/Read+split+pair_with_one+group/Reify+group/Write
root: INFO: 2018-03-21T17:01:01.659Z: JOB_MESSAGE_BASIC: Autoscaling: Resizing
worker pool from 1 to 2.
root: INFO: 2018-03-21T17:01:13.537Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised
the number of workers to 1 based on the rate of progress in the currently
running step(s).
root: INFO: 2018-03-21T17:01:13.564Z: JOB_MESSAGE_DETAILED: Resized worker pool
to 1, though goal was 2. This could be a quota issue.
root: INFO: 2018-03-21T17:01:29.274Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised
the number of workers to 2 based on the rate of progress in the currently
running step(s).
root: INFO: 2018-03-21T17:01:45.506Z: JOB_MESSAGE_DETAILED: Workers have
started successfully.
root: INFO: 2018-03-21T17:05:53.476Z: JOB_MESSAGE_BASIC: Executing operation
group/Close
root: INFO: 2018-03-21T17:05:54.513Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/DoOnce/Read.out" materialized.
root: INFO: 2018-03-21T17:05:54.537Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/MapToVoidKey0.out" materialized.
root: INFO: 2018-03-21T17:05:54.573Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0.out" materialized.
root: INFO: 2018-03-21T17:05:54.596Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/WriteBundles/MapToVoidKey0.out" materialized.
root: INFO: 2018-03-21T17:05:54.619Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)
root: INFO: 2018-03-21T17:05:54.866Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)
root: INFO: 2018-03-21T17:05:54.900Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)
root: INFO: 2018-03-21T17:05:54.927Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0).output"
materialized.
root: INFO: 2018-03-21T17:05:54.972Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0).output"
materialized.
root: INFO: 2018-03-21T17:05:54.995Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0).output"
materialized.
root: INFO: 2018-03-21T17:05:55.028Z: JOB_MESSAGE_BASIC: Executing operation
group/Read+group/GroupByWindow+count+format+write/Write/WriteImpl/WriteBundles/WriteBundles+write/Write/WriteImpl/Pair+write/Write/WriteImpl/WindowInto(WindowIntoFn)+write/Write/WriteImpl/GroupByKey/Reify+write/Write/WriteImpl/GroupByKey/Write
root: INFO: 2018-03-21T17:05:56.498Z: JOB_MESSAGE_ERROR: Traceback (most recent
call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 609, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 167, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 340, in
apache_beam.runners.worker.operations.DoOperation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 341, in
apache_beam.runners.worker.operations.DoOperation.start
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 373, in
apache_beam.runners.worker.operations.DoOperation.start
self.dofn_runner = common.DoFnRunner(
File "apache_beam/runners/common.py", line 483, in
apache_beam.runners.common.DoFnRunner.__init__
self.do_fn_invoker = DoFnInvoker.create_invoker(
File "apache_beam/runners/common.py", line 203, in
apache_beam.runners.common.DoFnInvoker.create_invoker
return PerWindowInvoker(
File "apache_beam/runners/common.py", line 313, in
apache_beam.runners.common.PerWindowInvoker.__init__
input_args, input_kwargs, [si[global_window] for si in side_inputs])
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/sideinputs.py",
line 62, in __getitem__
self._cache[window] = self._view_class._from_runtime_iterable(
AttributeError: type object '_DataflowIterableSideInput' has no attribute
'_from_runtime_iterable'
root: INFO: 2018-03-21T17:05:57.492Z: JOB_MESSAGE_ERROR: Traceback (most recent
call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 609, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 167, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 340, in
apache_beam.runners.worker.operations.DoOperation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 341, in
apache_beam.runners.worker.operations.DoOperation.start
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 373, in
apache_beam.runners.worker.operations.DoOperation.start
self.dofn_runner = common.DoFnRunner(
File "apache_beam/runners/common.py", line 483, in
apache_beam.runners.common.DoFnRunner.__init__
self.do_fn_invoker = DoFnInvoker.create_invoker(
File "apache_beam/runners/common.py", line 203, in
apache_beam.runners.common.DoFnInvoker.create_invoker
return PerWindowInvoker(
File "apache_beam/runners/common.py", line 313, in
apache_beam.runners.common.PerWindowInvoker.__init__
input_args, input_kwargs, [si[global_window] for si in side_inputs])
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/sideinputs.py",
line 62, in __getitem__
self._cache[window] = self._view_class._from_runtime_iterable(
AttributeError: type object '_DataflowIterableSideInput' has no attribute
'_from_runtime_iterable'
root: INFO: 2018-03-21T17:05:58.997Z: JOB_MESSAGE_ERROR: Traceback (most recent
call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 609, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 167, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 340, in
apache_beam.runners.worker.operations.DoOperation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 341, in
apache_beam.runners.worker.operations.DoOperation.start
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 373, in
apache_beam.runners.worker.operations.DoOperation.start
self.dofn_runner = common.DoFnRunner(
File "apache_beam/runners/common.py", line 483, in
apache_beam.runners.common.DoFnRunner.__init__
self.do_fn_invoker = DoFnInvoker.create_invoker(
File "apache_beam/runners/common.py", line 203, in
apache_beam.runners.common.DoFnInvoker.create_invoker
return PerWindowInvoker(
File "apache_beam/runners/common.py", line 313, in
apache_beam.runners.common.PerWindowInvoker.__init__
input_args, input_kwargs, [si[global_window] for si in side_inputs])
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/sideinputs.py",
line 62, in __getitem__
self._cache[window] = self._view_class._from_runtime_iterable(
AttributeError: type object '_DataflowIterableSideInput' has no attribute
'_from_runtime_iterable'
root: INFO: 2018-03-21T17:05:59.377Z: JOB_MESSAGE_ERROR: Traceback (most recent
call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 609, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 167, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 340, in
apache_beam.runners.worker.operations.DoOperation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 341, in
apache_beam.runners.worker.operations.DoOperation.start
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 373, in
apache_beam.runners.worker.operations.DoOperation.start
self.dofn_runner = common.DoFnRunner(
File "apache_beam/runners/common.py", line 483, in
apache_beam.runners.common.DoFnRunner.__init__
self.do_fn_invoker = DoFnInvoker.create_invoker(
File "apache_beam/runners/common.py", line 203, in
apache_beam.runners.common.DoFnInvoker.create_invoker
return PerWindowInvoker(
File "apache_beam/runners/common.py", line 313, in
apache_beam.runners.common.PerWindowInvoker.__init__
input_args, input_kwargs, [si[global_window] for si in side_inputs])
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/sideinputs.py",
line 62, in __getitem__
self._cache[window] = self._view_class._from_runtime_iterable(
AttributeError: type object '_DataflowIterableSideInput' has no attribute
'_from_runtime_iterable'
root: INFO: 2018-03-21T17:05:59.876Z: JOB_MESSAGE_ERROR: Traceback (most recent
call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 609, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 167, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 340, in
apache_beam.runners.worker.operations.DoOperation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 341, in
apache_beam.runners.worker.operations.DoOperation.start
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 373, in
apache_beam.runners.worker.operations.DoOperation.start
self.dofn_runner = common.DoFnRunner(
File "apache_beam/runners/common.py", line 483, in
apache_beam.runners.common.DoFnRunner.__init__
self.do_fn_invoker = DoFnInvoker.create_invoker(
File "apache_beam/runners/common.py", line 203, in
apache_beam.runners.common.DoFnInvoker.create_invoker
return PerWindowInvoker(
File "apache_beam/runners/common.py", line 313, in
apache_beam.runners.common.PerWindowInvoker.__init__
input_args, input_kwargs, [si[global_window] for si in side_inputs])
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/sideinputs.py",
line 62, in __getitem__
self._cache[window] = self._view_class._from_runtime_iterable(
AttributeError: type object '_DataflowIterableSideInput' has no attribute
'_from_runtime_iterable'
root: INFO: 2018-03-21T17:06:01.304Z: JOB_MESSAGE_ERROR: Traceback (most recent
call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 609, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 167, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 340, in
apache_beam.runners.worker.operations.DoOperation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 341, in
apache_beam.runners.worker.operations.DoOperation.start
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 373, in
apache_beam.runners.worker.operations.DoOperation.start
self.dofn_runner = common.DoFnRunner(
File "apache_beam/runners/common.py", line 483, in
apache_beam.runners.common.DoFnRunner.__init__
self.do_fn_invoker = DoFnInvoker.create_invoker(
File "apache_beam/runners/common.py", line 203, in
apache_beam.runners.common.DoFnInvoker.create_invoker
return PerWindowInvoker(
File "apache_beam/runners/common.py", line 313, in
apache_beam.runners.common.PerWindowInvoker.__init__
input_args, input_kwargs, [si[global_window] for si in side_inputs])
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/sideinputs.py",
line 62, in __getitem__
self._cache[window] = self._view_class._from_runtime_iterable(
AttributeError: type object '_DataflowIterableSideInput' has no attribute
'_from_runtime_iterable'
root: INFO: 2018-03-21T17:06:02.794Z: JOB_MESSAGE_ERROR: Traceback (most recent
call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 609, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 167, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 340, in
apache_beam.runners.worker.operations.DoOperation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 341, in
apache_beam.runners.worker.operations.DoOperation.start
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 373, in
apache_beam.runners.worker.operations.DoOperation.start
self.dofn_runner = common.DoFnRunner(
File "apache_beam/runners/common.py", line 483, in
apache_beam.runners.common.DoFnRunner.__init__
self.do_fn_invoker = DoFnInvoker.create_invoker(
File "apache_beam/runners/common.py", line 203, in
apache_beam.runners.common.DoFnInvoker.create_invoker
return PerWindowInvoker(
File "apache_beam/runners/common.py", line 313, in
apache_beam.runners.common.PerWindowInvoker.__init__
input_args, input_kwargs, [si[global_window] for si in side_inputs])
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/sideinputs.py",
line 62, in __getitem__
self._cache[window] = self._view_class._from_runtime_iterable(
AttributeError: type object '_DataflowIterableSideInput' has no attribute
'_from_runtime_iterable'
root: INFO: 2018-03-21T17:06:02.831Z: JOB_MESSAGE_DEBUG: Executing failure step
failure25
root: INFO: 2018-03-21T17:06:02.871Z: JOB_MESSAGE_ERROR: Workflow failed.
Causes:
S08:group/Read+group/GroupByWindow+count+format+write/Write/WriteImpl/WriteBundles/WriteBundles+write/Write/WriteImpl/Pair+write/Write/WriteImpl/WindowInto(WindowIntoFn)+write/Write/WriteImpl/GroupByKey/Reify+write/Write/WriteImpl/GroupByKey/Write
failed., A work item was attempted 4 times without success. Each time the
worker eventually lost contact with the service. The work item was attempted
on:
beamapp-jenkins-032117004-03211000-fff2-harness-trct,
beamapp-jenkins-032117004-03211000-fff2-harness-7f7d,
beamapp-jenkins-032117004-03211000-fff2-harness-7f7d,
beamapp-jenkins-032117004-03211000-fff2-harness-7f7d
root: INFO: 2018-03-21T17:06:02.981Z: JOB_MESSAGE_DETAILED: Cleaning up.
root: INFO: 2018-03-21T17:06:03.050Z: JOB_MESSAGE_DEBUG: Starting worker pool
teardown.
root: INFO: 2018-03-21T17:06:03.072Z: JOB_MESSAGE_BASIC: Stopping worker pool...
root: INFO: 2018-03-21T17:07:32.981Z: JOB_MESSAGE_DETAILED: Autoscaling:
Reduced the number of workers to 0 based on the rate of progress in the
currently running step(s).
root: INFO: 2018-03-21T17:07:33.014Z: JOB_MESSAGE_BASIC: Worker pool stopped.
root: INFO: 2018-03-21T17:07:33.422Z: JOB_MESSAGE_DEBUG: Tearing down pending
resources...
root: INFO: Job 2018-03-21_10_00_41-9840485252195089948 is in state
JOB_STATE_FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 3 tests in 424.622s
FAILED (errors=1)
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-03-21_10_00_41-9840485252195089948?project=apache-beam-testing
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-03-21_10_00_40-808610744177135573?project=apache-beam-testing
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-03-21_10_00_41-1656507014656882809?project=apache-beam-testing
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user hero...@google.com
Not sending mail to unregistered user aal...@gmail.com
