To fix your issues with the coder reference, you'll want to apply similar logic as WithKeys.java <https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/WithKeys.java#L114> which sets the coder on the output PCollection before returning it to the user.
You'll want to get the output type of the simple function and build a KV coder from it. The issue is about having a reliable way to get the output type of the simple function. On Tue, Jul 5, 2016 at 9:41 AM, Pawel Szczur <[email protected]> wrote: > Hi, > > Often I encounter a pattern: one gets a big data item, uses some part of > it as key, another as value. Thus I've created simple helper PTransform: > https://gist.github.com/orian/265e5494e61de931bbfa885ac60eb539 > > This allows me to reuse given function in manu places, sometimes as key > producer and another time as value producer. > > The usage is super simple: > PCollection<KV<String, String>> out = input.apply( > MakeKV.of(new SimpleFn(), new Simple2Fn())); > > In abstract it takes PCollection<V> and returns PCollection<KV<K,V1>>., > > BTW I had problems with: Cannot provide a Coder for type variable T > because the actual type is unknown due to erasure > > I've found: > 1. > http://stackoverflow.com/questions/32591914/making-transformations-in-dataflow-generic > 2. > http://stackoverflow.com/questions/32591914/making-transformations-in-dataflow-generic > but neither really made it easy to fix (I've been testing the MakeKV with > built in Integer type/coder). > > I've found a solution in WithKeys.java in SDK. > > Hope it helps someone. > > Cheers, Pawel >
