Re: [I] File not found while using metadata table for insert_overwrite table [hudi]
nsivabalan commented on issue #10628: URL: https://github.com/apache/hudi/issues/10628#issuecomment-2043996684 hey @ad1happy2go : if this turns out to be MDT data consistency issue, do keep me posted. thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] File not found while using metadata table for insert_overwrite table [hudi]
ad1happy2go commented on issue #10628: URL: https://github.com/apache/hudi/issues/10628#issuecomment-1980861181 @shravanak Are you still facing this issue? Let us know in case you need help here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] File not found while using metadata table for insert_overwrite table [hudi]
ad1happy2go commented on issue #10628: URL: https://github.com/apache/hudi/issues/10628#issuecomment-1943189279 @shravanak That may be the cause probably. Did you faced this issue with other tables also? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] File not found while using metadata table for insert_overwrite table [hudi]
shravanak commented on issue #10628: URL: https://github.com/apache/hudi/issues/10628#issuecomment-1933499929 We are using insert write mode with hudi 0.14.0 I think the file or partition it is referring to missing might be before we upgraded to 0.14.0 which was on 0.12.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] File not found while using metadata table for insert_overwrite table [hudi]
ad1happy2go commented on issue #10628: URL: https://github.com/apache/hudi/issues/10628#issuecomment-1933440282 @Shubham21k Code link here - https://gist.github.com/ad1happy2go/364e66c4fa84229110f28994cc4a277f Async services are meant to run with streaming workloads like Hudi Streamer, so that table services can run asynchronously and doesn't block the ingestion of next micro batch. Having it with Data source writers (batch writers) doesn't make any sense and inline table services will be kicked in. @shravanak Which Hudi version you are using? Are you also using insert_overwrite. Can you elaborate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] File not found while using metadata table for insert_overwrite table [hudi]
shravanak commented on issue #10628: URL: https://github.com/apache/hudi/issues/10628#issuecomment-1933190181 Is there a work around for this issue. We are facing a similar issue as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] File not found while using metadata table for insert_overwrite table [hudi]
Shubham21k commented on issue #10628: URL: https://github.com/apache/hudi/issues/10628#issuecomment-1931428449 @ad1happy2go 1. all the select queries are failing due to this error. As suggested, will try and check if this gets fixed in 0.14.1 2. 'async services doesn't work with Datasource writer' - can you elaborate more as we have observed the cleaner & archival taking place with these configurations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] File not found while using metadata table for insert_overwrite table [hudi]
ad1happy2go commented on issue #10628: URL: https://github.com/apache/hudi/issues/10628#issuecomment-1930407268 @Shubham21k What queries you are trying on this data? Does select * works? For point in time queries, this error is expected in case the commit is not archive but cleaned. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] File not found while using metadata table for insert_overwrite table [hudi]
ad1happy2go commented on issue #10628: URL: https://github.com/apache/hudi/issues/10628#issuecomment-1930403617 @Shubham21k Can you try with 0.14.1 once. Also, async services doesn't work with Datasource writer. I tried to reproduce this but unable to do it. Can you check in case you can enhance the same and reproduce it. Code here - https://gist.github.com/ad1happy2go/364e66c4fa84229110f28994cc4a277f/edit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
