In a nutshell, is this what you are trying to do?
1. Read the Delta table into a Spark DataFrame. 2. Explode the string column into a struct column. 3. Convert the hexadecimal field to an integer. 4. Write the DataFrame back to the Delta table in merge mode with a unique key. Is this a fair assessment HTH Mich Talebzadeh, Distinguished Technologist, Solutions Architect & Engineer London United Kingdom view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Mon, 9 Oct 2023 at 14:46, Karthick Nk <kcekarth...@gmail.com> wrote: > Hi All, > > I have mentioned the sample data below and the operation I need to > perform over there, > > I have delta tables with columns, in that columns I have the data in the > string data type(contains the struct data), > > So, I need to update one key value in the struct field data in the string > column of the delta table. > > Note: I can able to explode the string column into the struct field and > into the individual field by using the following operation in the spark, > > [image: image.png] > > df_new = spark.read.json(df.rdd.map(lambda x: '{"data": x.data }') > > Could you suggest a possible way to perform the required action in an > optimistic way? > > Note: Please feel free to ask, if you need further information. > > Thanks & regards, > Karthick > > On Mon, Oct 2, 2023 at 10:53 PM Karthick Nk <kcekarth...@gmail.com> wrote: > >> Hi community members, >> >> In databricks adls2 delta tables, I need to perform the below operation, >> could you help me with your thoughts >> >> I have the delta tables with one colum with data type string , which >> contains the json data in string data type, I need to do the following >> 1. I have to update one particular field value in the json and update it >> back in the same column of the data. >> >> Example : >> >> In string column, inside json I have one field with value in hexadecimal. >> Like { version : ''0xabcd1234"} >> >> I have to convert this field into corresponding integer value and update >> back into same strong column json value. >> Note: I have to perform this operation within this column. This column is >> basically with data type string in delta table. >> >> Could you suggest some sample example. >> >> Thanks in advance. >> >
