[GitHub] [hudi] wangxianghu commented on pull request #2993: [HUDI-1929] Make HoodieDeltaStreamer support configure KeyGenerator by type
wangxianghu commented on pull request #2993: URL: https://github.com/apache/hudi/pull/2993#issuecomment-853037788 > May I know what do you mean by "Yes, several places. I will change them one by one" ? Do you mean to say, you plan to address in a follow up patch ? IMO, it makes sense to do it in this patch itself. Ok, I'll update this pr to change all of them at once -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] wangxianghu commented on pull request #2993: [HUDI-1929] Make HoodieDeltaStreamer support configure KeyGenerator by type
wangxianghu commented on pull request #2993: URL: https://github.com/apache/hudi/pull/2993#issuecomment-851517633 > guess we need to fix quite a few more places. > https://github.com/apache/hudi/blob/e6a71ea544f3dd1ab5227b4c89a1540df5f7891a/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala#L213 > > https://github.com/apache/hudi/blob/e6a71ea544f3dd1ab5227b4c89a1540df5f7891a/hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/DataSourceUtils.java#L90 > > Also, I see you have added a keygen factory for spark. what's the plan for Flink ? Currently, flink and java engine shares key generator, so I planed to add a factory in `hudi-client-common` to support both of them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] wangxianghu commented on pull request #2993: [HUDI-1929] Make HoodieDeltaStreamer support configure KeyGenerator by type
wangxianghu commented on pull request #2993: URL: https://github.com/apache/hudi/pull/2993#issuecomment-851509812 > few Yes, several places. I will change them one by one -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] wangxianghu commented on pull request #2993: [HUDI-1929] Make HoodieDeltaStreamer support configure KeyGenerator by type
wangxianghu commented on pull request #2993: URL: https://github.com/apache/hudi/pull/2993#issuecomment-851507979 > guess we need to fix quite a few more places. > https://github.com/apache/hudi/blob/e6a71ea544f3dd1ab5227b4c89a1540df5f7891a/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala#L213 > > https://github.com/apache/hudi/blob/e6a71ea544f3dd1ab5227b4c89a1540df5f7891a/hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/DataSourceUtils.java#L90 > > Also, I see you have added a keygen factory for spark. what's the plan for Flink ? The keyGenerator of flink is in `hudi-client-common`, I plan to add a factory for flink and java engine. they can share it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] wangxianghu commented on pull request #2993: [HUDI-1929] Make HoodieDeltaStreamer support configure KeyGenerator by type
wangxianghu commented on pull request #2993: URL: https://github.com/apache/hudi/pull/2993#issuecomment-848482449 @yanghua please take a look when free -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
