Re: [I] [SUPPORT] Async Clustering: Seeking Help on Specific Partitioning and Regex Pattern [hudi]

[via GitHub]

noahtaite commented on issue #10165:
URL: https://github.com/apache/hudi/issues/10165#issuecomment-1828595136

   > <img alt="Screenshot 2023-11-27 at 9 07 05 AM" width="285" 
src="https://private-user-images.githubusercontent.com/39345855/285898963-1b019a83-1924-441f-bf07-a1816aea2b93.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTEiLCJleHAiOjE3MDExMTgwMzMsIm5iZiI6MTcwMTExNzczMywicGF0aCI6Ii8zOTM0NTg1NS8yODU4OTg5NjMtMWIwMTlhODMtMTkyNC00NDFmLWJmMDctYTE4MTZhZWEyYjkzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFJV05KWUFYNENTVkVINTNBJTJGMjAyMzExMjclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjMxMTI3VDIwNDIxM1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTczYWVhM2E2OWNlMzMwYzg1Mzk2YWEyODZkYTIyODkxYmEzMmMwOWZmOGM4ZmE4ZWIzZjBkNzFhYTczMDBkZmEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.KNfr2st0vjAD46TUdp8tCRE4gEH0fA9s0Jf5rHoYpAU";>
   
   Hey @soumilshah1995 can we see the output of 
df.select("_hoodie_partition_path")? To me it appears as though your data uses 
the partition fields "year,month" given that your data is structured like:
   ```
   orders/2023/10
   orders/2023/11
   ```
   
   So I think partition_path may be like "2023/10" + "2023/11". Which could 
explain why it is not getting selected as you expect. From my experience 
working with Hudi services that rely on partition path, they usually expect it 
in the format that the data is physically written (which can be figured out by 
looking at the `_hoodie_partition_path` field).
   
   Note I am also a Hudi user, so just trying to help given the information 
presented.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org