damccorm commented on issue #21444: URL: https://github.com/apache/beam/issues/21444#issuecomment-1238586362
> Should we make returning _just_ the inference the default and handle this with a composable `ModelHandler` that produces `PredictionResult` instances, similar to `KeyedModelHandler`? > > Unfortunately that couldn't be done in a backwards compatible way. My instinct is that the large majority of users are going to want both passed through and we should prioritize that need (and not making a breaking change) over making things a little cleaner for the just inference case. The other advantage of always returning a PredictionResult is that it gives users more flexibility if, for example, they initially only need the inferences but later want to do something with the input itself. If we just return the inferences then they need to insert some intermediate mapping step to make their existing logic work, if we always return a `PredictionResult` then they can flip the flag, grab the initial input wherever they need it, and not touch anything else. ---------- There's also a larger philosophical question of when we should introduce a new model handler vs parameterizing a new behavior option. I'd argue that because in this case omitting the input is essentially a subset of the current output (with no changes to the input) and we're not changing the graph structure we're better off treating this as an option (parameter to the `RunInference` call) than introducing a new handler. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
