Hi Allen, You are able to configure the key generator for deltastreamer using this property (either via a file or --config ) hoodie.datasource.write.keygenerator.class
You might be interested in this built-in generator. https://github.com/apache/incubator-hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/keygen/TimestampBasedKeyGenerator.java#L64 It takes let you configure a field as a recordKey, and if you can parse your timestamp using Java SimpleDateFormat, you can specify the datetime field and a pattern to parse it into.. Happy to make this work for you. community, any volunteers to faq/document this? :) On Mon, May 4, 2020 at 9:11 AM Allen Underwood <[email protected]> wrote: > I’ve tried to do my due diligence by googling / searching this slack and > I’ve come up empty.Is there a way through configuration / deltastreamer > to extract a custom partition key?Basically I have a datetime field in a > Kafka Source that has an ISO8601 datetime….is there a way to extract a > partition value out of that?I found this after some Googling, but this > seems like it’d only be useful if I wanted to write my own writer > application: > > https://github.com/apache/incubator-hudi/blob/master/hudi-spark/src/main/java/org/apache/hudi/keygen/ComplexKeyGenerator.javaAny > way to do what I need through configuration of the spark job / hudi > configuration? > hudi-spark/src/main/java/org/apache/hudi/keygen/ComplexKeyGenerator.java > <https://github.com/apache/incubator-hudi/blob/master/hudi-spark/src/main/java/org/apache/hudi/keygen/ComplexKeyGenerator.java> > > -- > *Allen Underwood* >
