Hi Vinoth, I have already raised a lira for this some time back - https://issues.apache.org/jira/browse/HUDI-859. I am waiting for https://github.com/apache/incubator-hudi/pull/1433 to be closed before I can start working on it. :)
On Mon, May 4, 2020 at 9:53 PM Vinoth Chandar <[email protected]> wrote: > Hi Allen, > > You are able to configure the key generator for deltastreamer using this > property (either via a file or --config ) > hoodie.datasource.write.keygenerator.class > > You might be interested in this built-in generator. > > https://github.com/apache/incubator-hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/keygen/TimestampBasedKeyGenerator.java#L64 > It takes let you configure a field as a recordKey, and if you can parse > your timestamp using Java SimpleDateFormat, you can specify the datetime > field and a pattern to parse it into.. > > Happy to make this work for you. > > community, any volunteers to faq/document this? :) > > > On Mon, May 4, 2020 at 9:11 AM Allen Underwood > <[email protected]> wrote: > > > I’ve tried to do my due diligence by googling / searching this slack and > > I’ve come up empty.Is there a way through configuration / deltastreamer > > to extract a custom partition key?Basically I have a datetime field in a > > Kafka Source that has an ISO8601 datetime….is there a way to extract a > > partition value out of that?I found this after some Googling, but this > > seems like it’d only be useful if I wanted to write my own writer > > application: > > > > > https://github.com/apache/incubator-hudi/blob/master/hudi-spark/src/main/java/org/apache/hudi/keygen/ComplexKeyGenerator.javaAny > > way to do what I need through configuration of the spark job / hudi > > configuration? > > hudi-spark/src/main/java/org/apache/hudi/keygen/ComplexKeyGenerator.java > > < > https://github.com/apache/incubator-hudi/blob/master/hudi-spark/src/main/java/org/apache/hudi/keygen/ComplexKeyGenerator.java > > > > > > -- > > *Allen Underwood* > > >
