[GitHub] [hudi] yihua commented on pull request #1149: [HUDI-472] Introduce configurations and new modes of sorting for bulk_insert

2020-07-20 Thread GitBox
yihua commented on pull request #1149: URL: https://github.com/apache/hudi/pull/1149#issuecomment-661421240 @nsivabalan Thanks for the fix. There is some ad-hoc code in this PR just for testing. Let me clean that up. This

[GitHub] [hudi] yihua commented on pull request #1149: [HUDI-472] Introduce configurations and new modes of sorting for bulk_insert

2020-07-23 Thread GitBox
yihua commented on pull request #1149: URL: https://github.com/apache/hudi/pull/1149#issuecomment-663276947 I also simplified the naming of the sorting modes to: `NONE`, `GLOBAL_SORT`, and `PARTITION_SORT`. This is an automa

[GitHub] [hudi] yihua commented on pull request #1149: [HUDI-472] Introduce configurations and new modes of sorting for bulk_insert

2020-07-29 Thread GitBox
yihua commented on pull request #1149: URL: https://github.com/apache/hudi/pull/1149#issuecomment-664834387 @vinothchandar @nsivabalan this PR is ready for another review. I fixed the failing tests. I also simplified the bulk insert logic regarding different sort modes. Besides, I

[GitHub] [hudi] yihua commented on pull request #1149: [HUDI-472] Introduce configurations and new modes of sorting for bulk_insert

2020-07-30 Thread GitBox
yihua commented on pull request #1149: URL: https://github.com/apache/hudi/pull/1149#issuecomment-666757074 I'll take a pass tonight. Also, @nsivabalan could you `Squash and merge` in Github? I'd like to keep the commits local in this PR.

[GitHub] [hudi] yihua commented on pull request #1149: [HUDI-472] Introduce configurations and new modes of sorting for bulk_insert

2020-07-31 Thread GitBox
yihua commented on pull request #1149: URL: https://github.com/apache/hudi/pull/1149#issuecomment-667345505 @nsivabalan @n3nash I just recall that Uber internally extends `UserDefinedBulkInsertPartitioner` to have custom partitioners so the name change of the `UserDefinedBulkInsertPartiti