byronellis commented on code in PR #28289:
URL: https://github.com/apache/beam/pull/28289#discussion_r1328944486
##########
sdks/python/apache_beam/yaml/yaml_provider.py:
##########
@@ -61,6 +61,16 @@ def provided_transforms(self) -> Iterable[str]:
"""Returns a list of transform type names this provider can handle."""
raise NotImplementedError(type(self))
+ def requires_inputs(self, typ: str, args: Mapping[str, Any]) -> bool:
+ """Returns whether this transform requires inputs.
+
+ Specifically, if this returns True and inputs are not provided than an
error
+ will be thrown.
+
+ This is best-effort, primarily for better and earlier error messages.
+ """
+ return not typ.startswith('Read')
Review Comment:
Presuming this is to provide a reasonable default? Seems like it might lead
to surprising false positives though?
##########
sdks/python/apache_beam/yaml/yaml_provider.py:
##########
@@ -138,8 +146,19 @@ def create_transform(self, type, args,
yaml_create_transform):
except Exception:
# It's possible this service doesn't vend schema transforms.
self._schema_transforms = {}
+ return self._schema_transforms
+
+ def requires_inputs(self, typ, args):
+ if self._urns[type] in self.schema_transforms():
+ return bool(self.schema_transforms()[self._urns[type]].inputs)
Review Comment:
Unfortunately relying on schema transform's inputs/outputs is not very
reliable (and maybe completely unreliable). They do not depend on the
configuration passed to the transforms making them basically useless for
determining anything other than inputs/outputs that are present in all possible
configurations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
