[
https://issues.apache.org/jira/browse/BEAM-7629?focusedWorklogId=266267&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-266267
]
ASF GitHub Bot logged work on BEAM-7629:
----------------------------------------
Author: ASF GitHub Bot
Created on: 25/Jun/19 01:35
Start Date: 25/Jun/19 01:35
Worklog Time Spent: 10m
Work Description: youngoli commented on issue #8936: [BEAM-7629] Go SDK
additional Validation for DoFns (1st impl) (DO NOT MERGE)
URL: https://github.com/apache/beam/pull/8936#issuecomment-505241954
R: @lostluck
Hoping I could get your feedback on what I have so far and help answering
some questions.
1. Some of my tests currently fail because the following is expected in
signatures for all DoFn methods: `side input parameters must follow main input
parameter`. My understanding however is that when including side inputs in the
StartBundle and FinishBundle methods (if ProcessElement has some) only the side
inputs should be included, with no main inputs. So theoretically it should
work, but right now all methods are treated as if they're ProcessElement. Is
this correct?
2. I had trouble finding info for the expected inputs and outputs of the
DoFn methods, and had to resort to checking other SDKs. Is there a place where
that's documented for the Go SDK? If not, could you clarify for StartBundle,
FinishBundle, Setup, and Teardown what parameters and return values they cab
have (in addition to the side inputs and emits that I already account for)?
Right now I make an assumption that Setup/Teardown shouldn't have any
parameters based on the Java SDK, but couldn't find any concrete info for
return values for any of the methods.
3. Should I worry about the Go SDK "generics" when it comes to the side
input and emit validation? Is it even valid for a user to use generics to
replace emits or side inputs there, and if so should I be checking that they're
consistent? For this first draft I just ignored the possibility of generics.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 266267)
Time Spent: 20m (was: 10m)
> Improve DoFn method validation in core/graph/fn.go
> --------------------------------------------------
>
> Key: BEAM-7629
> URL: https://issues.apache.org/jira/browse/BEAM-7629
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Daniel Oliveira
> Assignee: Daniel Oliveira
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Various improvements can be made to validating the signatures and type usages
> in DoFns. Some things that should probably be checked:
> * Check that StartBundle and FinishBundle contain any emit parameters andÂ
> side inputs present in ProcessElement
> * Check that any side inputs/emits have correctly matching types between
> Start/FinishBundle and ProcessElement
> * Check that parameters and return values for the various methods are valid
> (for ex. Teardown/Setup should have no params)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
