Re: Error while merge in delta table

2023-05-12 Thread Farhan Misarwala
Hi Karthick, If you have confirmed that the incompatibility between Delta and spark versions is not the case, then I would say the same what Jacek said earlier, there’s not enough “data” here. To further comment on it, we would need to know more on how you are structuring your multi threaded PySp

Re: Error while merge in delta table

2023-05-12 Thread Karthick Nk
Hi Farhan, Thank you for your response, I am using databricks with 11.3x-scala2.12. Here I am overwriting all the tables in the same database in concurrent thread, But when I do in the iterative manner it is working fine, For Example, i am having 200 tables in same database, i am overwriting the

Re: Error while merge in delta table

2023-05-11 Thread Farhan Misarwala
Hi Karthick, I think I have seen this before and this probably could be because of an incompatibility between your spark and delta versions. Or an incompatibility between the delta version you are using now vs the one you used earlier on the existing table you are merging with. Let me know if th

Re: Error while merge in delta table

2023-05-11 Thread Jacek Laskowski
Hi Karthick, Sorry to say it but there's not enough "data" to help you. There should be something more above or below this exception snippet you posted that could pinpoint the root cause. Pozdrawiam, Jacek Laskowski "The Internals Of" Online Books Follow me on http

Error while merge in delta table

2023-05-10 Thread Karthick Nk
Hi, I am trying to merge daaframe with delta table in databricks, but i am getting error, i have attached the code nippet and error message for reference below, code: [image: image.png] error: [image: image.png] Thanks