Re: Implementing type hints on multi-output PTransforms

2020-04-13 Thread Udi Meiri
On Tue, Mar 31, 2020 at 10:16 AM Joshua B. Harrison wrote: > Ok - that makes sense. My specific workaround was to remove the > with_output_types for now, so advising the user on this in the error > message would be nice. I was just worried about silently passing. > > As for the formalization: >

Re: Implementing type hints on multi-output PTransforms

2020-03-31 Thread Joshua B. Harrison
Ok - that makes sense. My specific workaround was to remove the with_output_types for now, so advising the user on this in the error message would be nice. I was just worried about silently passing. As for the formalization: 1. I am a little confused on how this is different than passing

Re: Implementing type hints on multi-output PTransforms

2020-03-31 Thread Udi Meiri
Hi Joshua, I've been working on type hints recently. Your issue is similar to this: https://issues.apache.org/jira/browse/BEAM-8782 (exactly the same if tags are passed to with_outputs() in the example). There's also this related bug about type inference:

Re: Implementing type hints on multi-output PTransforms

2020-03-31 Thread Luke Cwik
I can see that argument but what does a user need to do in this case if we raise NotImplementedError? Would the need to disable type checking everywhere? Over the long term users will need to deal with improvements to type checking and will need to fix typing errors when they change Apache Beam

Re: Implementing type hints on multi-output PTransforms

2020-03-31 Thread Joshua B. Harrison
The current code errors out with a cryptic message around tag types in the multi-output. Adding a NotImplementedError was just an attempt to make the failure reason more clear. I would be worried about trivially passing because then the user might think they have type checking safety when they

Re: Implementing type hints on multi-output PTransforms

2020-03-31 Thread Luke Cwik
Would the NotImplementedError cause users pipeline errors or is that a signal to the type checking mechanism to ignore it? If this would cause failures I would rather make the unsupported case return something that would be trivially true. On Mon, Mar 30, 2020 at 12:01 PM Joshua B. Harrison

Implementing type hints on multi-output PTransforms

2020-03-30 Thread Joshua B. Harrison
Hey all, I brought up an issue recently on the user forums noting issues around type hints and multi-output PTransforms: https://lists.apache.org/thread.html/r94bf2e43f09a290dbe87d5a8d7eedb34ea215e0bea861521cbdb0c1c%40%3Cuser.beam.apache.org%3E As mentioned there, I think that a