KevinGG commented on code in PR #17736:
URL: https://github.com/apache/beam/pull/17736#discussion_r883124064
##########
sdks/python/apache_beam/runners/interactive/interactive_beam.py:
##########
@@ -418,15 +424,19 @@ def create(
raise ValueError(
'Unknown cluster identifier: %s. Cannot create or reuse'
'a Dataproc cluster.')
- elif cluster_metadata.region == 'global':
- # The global region is unsupported as it will be eventually deprecated.
- raise ValueError('Clusters in the global region are not supported.')
- elif not cluster_metadata.region:
+ if not cluster_metadata.region:
_LOGGER.info(
'No region information was detected, defaulting Dataproc cluster '
'region to: us-central1.')
cluster_metadata.region = 'us-central1'
+ elif cluster_metadata.region == 'global':
+ # The global region is unsupported as it will be eventually deprecated.
+ raise ValueError('Clusters in the global region are not supported.')
# else use the provided region.
+ if cluster_metadata.num_workers and cluster_metadata.num_workers < 2:
Review Comment:
The magic number is required by Dataproc but no documented. To avoid failing
the cluster creation with such an error, we do this check and override early
for them.
I'll move it to a constant with comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
