[GitHub] [hudi] bvaradar commented on issue #2269: [SUPPORT] - HUDI Table Bulk Insert for 5 gb parquet file progressively taking longer time to insert.

2020-12-02 Thread GitBox
bvaradar commented on issue #2269: URL: https://github.com/apache/hudi/issues/2269#issuecomment-737415489 There is an email thread around 0.7.0 release in users@ mailing list. It is expected to be by end of this month. Regarding inserts, Hudi takes care of automatically growing

[GitHub] [hudi] bvaradar commented on issue #2269: [SUPPORT] - HUDI Table Bulk Insert for 5 gb parquet file progressively taking longer time to insert.

2020-11-30 Thread GitBox
bvaradar commented on issue #2269: URL: https://github.com/apache/hudi/issues/2269#issuecomment-735921130 @asharma4-lucid : Sorry for the delay in responding due to Thanksgiving weekend. It looks like cleaning is the one taking long time. Cleaner (in 0.6) runs in incremental mode by

[GitHub] [hudi] bvaradar commented on issue #2269: [SUPPORT] - HUDI Table Bulk Insert for 5 gb parquet file progressively taking longer time to insert.

2020-11-24 Thread GitBox
bvaradar commented on issue #2269: URL: https://github.com/apache/hudi/issues/2269#issuecomment-733299492 @asharma4-lucid : ~5hrs is way too much. Can you disable cleaning using the config hoodie.clean.automatic=false and try. Is this a COW table ?

[GitHub] [hudi] bvaradar commented on issue #2269: [SUPPORT] - HUDI Table Bulk Insert for 5 gb parquet file progressively taking longer time to insert.

2020-11-23 Thread GitBox
bvaradar commented on issue #2269: URL: https://github.com/apache/hudi/issues/2269#issuecomment-732367698 1. Only the files having records to be updated or added will be "touched" by Hudi. 2.Hudi write operations would only load the partitions that are needed for writing (only

[GitHub] [hudi] bvaradar commented on issue #2269: [SUPPORT] - HUDI Table Bulk Insert for 5 gb parquet file progressively taking longer time to insert.

2020-11-21 Thread GitBox
bvaradar commented on issue #2269: URL: https://github.com/apache/hudi/issues/2269#issuecomment-731656591 @AakashPradeep : I can quickly tell that the number of partitions is really high relative to the file size in each partition. It looks like each partition has only very little records