The problem with being on vacation for a week is so many cool things come up while you're gone that you can't get ahead of...
Thank you Danny for the initial write up! I hope the exercise helped you understand more about the SDK and Go, but I think there's more to do here. *tl;dr; * 1. I don't think the doc went far enough. See a working sketch of my thoughts here: https://go.dev/play/p/Bwf4eSavNxt?v=gotip 2. Agreed that supporting generic KVs should still be possible, but it's an orthogonal investigation to the code generator (and its generated code) question. I won't discuss that further, as I have a different one-pager in the works on that topic, and that's a very very different thread. *On the Code Generator* Overall, the goal is to totally do-away with the current approach of pre-generating. But I don't think a lack of inference is an obstacle to that goal. I agree that it's going to be considered inconvenient for users to manually specify types at this stage. I think it's a substantial improvement over the status quo, simply because it removes a step in the iteration cycle when a change needs to be made. It has the compiler do the work for us mostly. The doc covers it's explored options very well, but it leaves out the possibility of getting halfway there: https://go.dev/play/p/Bwf4eSavNxt?v=gotip To cover the varieties of expected arities, we should generate the SDK side registration code in order to avoid missing cases, and ensure even coverage. And finally, even if inference is ultimately the sticking point: We should still rewrite the code generator to generate generic code for us, since it moves the harder parts (de-duplication, name collisions in types if used in both package code, and test code) into the Go compiler, rather than in our code. And who knows? Perhaps in Go 1.19 or 1.20, the inferences Danny proposed become possible, and we can clean things up further. Robert Burke, Beam Go Busybody On Fri, 11 Mar 2022 at 06:03, Danny McCormick <[email protected]> wrote: > > Can generics be used to at least get rid of the oddities of KV handling? > > Good question - my honest answer is that I don't know: this > doc/investigation was specifically focused on replacing the code generator > and generics not being useful in this context does not mean they can't be > useful elsewhere. > > My speculative answer (without having a full familiarity with how we > currently handle KVs or the associated problems) is that I don't *think *the > core blocker keeping us from replacing the code generator applies here. The > big issue we're facing here is that we can't infer generic interface types > based on the parameters of method functions of a struct. In the case of KV, > we would be inferring the interface types based on the types of member > variables of the struct, which Go should be able to do. > > All that to say, in no way should this investigation preclude us from > looking for other ways to apply generics to make our lives easier (whether > that be in KV handling or improving other experiences). > > On Thu, Mar 10, 2022 at 6:55 PM Robert Bradshaw <[email protected]> > wrote: > >> Thanks for the writeup. That's too bad. Can generics be used to at least >> get rid of the oddities of KV handling? >> >> On Thu, Mar 10, 2022 at 3:40 PM Luke Cwik <[email protected]> wrote: >> >>> Well written doc and unfortunate that Generics seem to not work out for >>> us. >>> >>> >>> >>> On Thu, Mar 10, 2022 at 11:03 AM Jack McCluskey <[email protected]> >>> wrote: >>> >>>> Well, looks like Go generics weren't quite the magic bullet we were >>>> hoping for in their current state. We'll get that code generator ripped out >>>> one day, but not today. >>>> >>>> On Thu, Mar 10, 2022 at 12:44 PM Danny McCormick < >>>> [email protected]> wrote: >>>> >>>>> Hey folks, >>>>> >>>>> Right now, the Go Sdk relies on users to generate code as a >>>>> pre-compile step to avoid some overhead of excessive runtime reflection. >>>>> This is suboptimal because it relies on users to regenerate code every >>>>> time >>>>> they create a DoFn, causing friction to users and slowness when they don't >>>>> remember to regenerate (among other reasons). @lostluck wrote a one pager >>>>> around replacing this code generator with generics - after spending pieces >>>>> of the last couple days digging in my takeaway is that it's not doable in >>>>> a >>>>> way that benefits the Sdk at this time and I wanted to share my findings. >>>>> I >>>>> wrote them up here - >>>>> https://docs.google.com/document/d/1imYbBeu2FNJkwPNm6E9GEJkjpHnHscvFoKAE6AISvFA/edit?usp=sharing >>>>> - >>>>> please take a look if you're interested, and definitely let me know if I'm >>>>> missing anything! >>>>> >>>>> Thanks, >>>>> Danny >>>>> >>>>
