[GitHub] [hudi] c-f-cooper commented on issue #8651: [SUPPORT]How to resolve small file?

2023-06-11 Thread via GitHub
c-f-cooper commented on issue #8651: URL: https://github.com/apache/hudi/issues/8651#issuecomment-1586527634 > The file sizing should be still valid. I try it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [hudi] c-f-cooper commented on issue #8651: [SUPPORT]How to resolve small file?

2023-06-11 Thread via GitHub
c-f-cooper commented on issue #8651: URL: https://github.com/apache/hudi/issues/8651#issuecomment-1586428325 > Hey there try adding this config i have tried and works for me > > ``` > # Define a dictionary with the Fiel Size > hudi_file_size = { > "hoodie.parqu

[GitHub] [hudi] c-f-cooper commented on issue #8651: [SUPPORT]How to resolve small file?

2023-05-18 Thread via GitHub
c-f-cooper commented on issue #8651: URL: https://github.com/apache/hudi/issues/8651#issuecomment-1553931467 ![image](https://github.com/apache/hudi/assets/25735549/8b816399-ede9-4a2a-97b5-d28e7ef3b1e4) ![023D4646-7D12-4606-8188-0F1A05DE47C5_1_102_o](https://github.com/apache/hudi/as

[GitHub] [hudi] c-f-cooper commented on issue #8651: [SUPPORT]How to resolve small file?

2023-05-07 Thread via GitHub
c-f-cooper commented on issue #8651: URL: https://github.com/apache/hudi/issues/8651#issuecomment-1537736137 yes,It has come into effect,the clustering task appears in DAG,and the replace_commit appears in .hoodie directory. -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] c-f-cooper commented on issue #8651: [SUPPORT]How to resolve small file?

2023-05-06 Thread via GitHub
c-f-cooper commented on issue #8651: URL: https://github.com/apache/hudi/issues/8651#issuecomment-1537307015 > > Do you do not enable the async clustering right? We have inline clustering, async clustering, and offline clustering, which one are you using? > > we use async clustering,w

[GitHub] [hudi] c-f-cooper commented on issue #8651: [SUPPORT]How to resolve small file?

2023-05-06 Thread via GitHub
c-f-cooper commented on issue #8651: URL: https://github.com/apache/hudi/issues/8651#issuecomment-1537306637 > Do you do not enable the async clustering right? We have inline clustering, async clustering, and offline clustering, which one are you using? we use async clustering,we use

[GitHub] [hudi] c-f-cooper commented on issue #8651: [SUPPORT]How to resolve small file?

2023-05-06 Thread via GitHub
c-f-cooper commented on issue #8651: URL: https://github.com/apache/hudi/issues/8651#issuecomment-1537144673 > @c-f-cooper Are you using Insert or Bulk Insert. Can you please share the clustering command and table configs you are using. we use cow+insert mode,beside the clustering con

[GitHub] [hudi] c-f-cooper commented on issue #8651: [SUPPORT]How to resolve small file?

2023-05-06 Thread via GitHub
c-f-cooper commented on issue #8651: URL: https://github.com/apache/hudi/issues/8651#issuecomment-1537144331 > What kind of clustering are you using, online or offline? For no effect do you mean no bigger Parquets are generated and replacing the existing files? we use online async clu