Re: [PR] [HUDI-1881]: draft implementation for trigger based on data availability [hudi]

2024-04-29 Thread via GitHub
Sarfaraz-214 commented on code in PR #5071: URL: https://github.com/apache/hudi/pull/5071#discussion_r1584145214 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieMultiTableDeltaStreamer.java: ## @@ -425,15 +432,29 @@ private static String resetTarget

Re: [PR] [HUDI-1881]: draft implementation for trigger based on data availability [hudi]

2024-01-09 Thread via GitHub
Sarfaraz-214 commented on PR #5071: URL: https://github.com/apache/hudi/pull/5071#issuecomment-1883033907 I cherry picked this commit into Hudi 0.14.1 and did some minor changes and it seems to working fine for me. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] [HUDI-1881]: draft implementation for trigger based on data availability [hudi]

2024-01-07 Thread via GitHub
Sarfaraz-214 commented on PR #5071: URL: https://github.com/apache/hudi/pull/5071#issuecomment-1880515749 @nsivabalan Siva, it seems @pratyakshsharma has already made an attempt at addressing the issue in the PR. Given the importance of the use-case and the progress made so far, it would be

Re: [PR] [HUDI-1881]: draft implementation for trigger based on data availability [hudi]

2024-01-03 Thread via GitHub
pratyakshsharma commented on PR #5071: URL: https://github.com/apache/hudi/pull/5071#issuecomment-1876489891 @nsivabalan I am willing to take it up. I have already given it a shot in this PR. Let me know if you wish to discuss on slack or here itself about the solution. -- This is an aut

Re: [PR] [HUDI-1881]: draft implementation for trigger based on data availability [hudi]

2024-01-03 Thread via GitHub
nsivabalan commented on PR #5071: URL: https://github.com/apache/hudi/pull/5071#issuecomment-1875829707 nope. I don't think we have a fix for that yet. let us know if you are interested. If you would like to contribute, we can guide w/ the solution and review the patch and assist w/ landing

Re: [PR] [HUDI-1881]: draft implementation for trigger based on data availability [hudi]

2023-12-26 Thread via GitHub
pratyakshsharma commented on PR #5071: URL: https://github.com/apache/hudi/pull/5071#issuecomment-1869535161 @nsivabalan should be able to share the latest on this I believe. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] [HUDI-1881]: draft implementation for trigger based on data availability [hudi]

2023-12-26 Thread via GitHub
Sarfaraz-214 commented on PR #5071: URL: https://github.com/apache/hudi/pull/5071#issuecomment-1869456574 Hi @pratyakshsharma @nsivabalan Can anyone let me know about the latest update on this limitation of HoodieMultiTableStreamer ? Where running HoodieMultiTableStreamer in --continuou