Re: [I] [SUPPORT] Hudi fails ACID verification test [hudi]

2024-05-17 Thread via GitHub
ad1happy2go commented on issue #11170: URL: https://github.com/apache/hudi/issues/11170#issuecomment-2117238692 Thanks @matthijseikelenboom for the update -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [I] [SUPPORT] Hudi fails ACID verification test [hudi]

2024-05-17 Thread via GitHub
matthijseikelenboom closed issue #11170: [SUPPORT] Hudi fails ACID verification test URL: https://github.com/apache/hudi/issues/11170 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] [SUPPORT] Hudi fails ACID verification test [hudi]

2024-05-17 Thread via GitHub
matthijseikelenboom commented on issue #11170: URL: https://github.com/apache/hudi/issues/11170#issuecomment-2117016086 Tested and verified. Closing issues. More info Solution has been tested on: - Java 8 ✅ - Java 11 ✅ - Java 17 ❌ (As of this moment, Hudi doesn't suppo

Re: [I] [SUPPORT] Hudi fails ACID verification test [hudi]

2024-05-16 Thread via GitHub
ad1happy2go commented on issue #11170: URL: https://github.com/apache/hudi/issues/11170#issuecomment-2115400503 @matthijseikelenboom I was able to successfully test. There were two issues - 1. InprocessLockProvider doesn't work for multiple writes. So use FileSystemBasedLockProvider in

Re: [I] [SUPPORT] Hudi fails ACID verification test [hudi]

2024-05-16 Thread via GitHub
ad1happy2go commented on issue #11170: URL: https://github.com/apache/hudi/issues/11170#issuecomment-2115401719 @matthijseikelenboom Please let us know if it works for you also. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] [SUPPORT] Hudi fails ACID verification test [hudi]

2024-05-16 Thread via GitHub
ad1happy2go commented on issue #11170: URL: https://github.com/apache/hudi/issues/11170#issuecomment-2115007277 @matthijseikelenboom I tried to run in my local but again seeing issues. We can connect once. If you are on Apache Hudi slack can you ping me "Aditya Goenka" -- This is an auto

Re: [I] [SUPPORT] Hudi fails ACID verification test [hudi]

2024-05-09 Thread via GitHub
matthijseikelenboom commented on issue #11170: URL: https://github.com/apache/hudi/issues/11170#issuecomment-2102747936 @ad1happy2go I've pushed a new branch on the repo where the project is downgraded to Java 8. When running the test then, the writers don't seem to fail anymore, but it sti

Re: [I] [SUPPORT] Hudi fails ACID verification test [hudi]

2024-05-09 Thread via GitHub
matthijseikelenboom commented on issue #11170: URL: https://github.com/apache/hudi/issues/11170#issuecomment-2102603694 Okay, yeah sure. The original test was written with Java 11, but I updated to 17 because I thought why not and Spark 3.4.2 supports it. Is it known that Hudi (Or Kry

Re: [I] [SUPPORT] Hudi fails ACID verification test [hudi]

2024-05-09 Thread via GitHub
ad1happy2go commented on issue #11170: URL: https://github.com/apache/hudi/issues/11170#issuecomment-2102355732 @matthijseikelenboom I noticed you are using JAVA 17 for the same. Hudi 0.14.1 doesn't support JAVA 17 yet. The newer Hudi version will be able to support the same. Some re

Re: [I] [SUPPORT] Hudi fails ACID verification test [hudi]

2024-05-08 Thread via GitHub
ad1happy2go commented on issue #11170: URL: https://github.com/apache/hudi/issues/11170#issuecomment-2101987758 @matthijseikelenboom Looks like some library conflicts are there in the project. Need to reproduce it. -- This is an automated message from the Apache Git Service. To respond t

Re: [I] [SUPPORT] Hudi fails ACID verification test [hudi]

2024-05-08 Thread via GitHub
matthijseikelenboom commented on issue #11170: URL: https://github.com/apache/hudi/issues/11170#issuecomment-2100205976 @ad1happy2go Ah yes, you're right. I seem to have forgot to add the hudi-defaults.conf file to this project. I've added it to my repository and ran the test again. It come

Re: [I] [SUPPORT] Hudi fails ACID verification test [hudi]

2024-05-08 Thread via GitHub
ad1happy2go commented on issue #11170: URL: https://github.com/apache/hudi/issues/11170#issuecomment-2100011078 @matthijseikelenboom I don't see any lock related configurations in your setup. I checked that you are using 2 parallel writers. So you may need to configure lock during write. Hu