Encoded data retrieved when reading Parquet file

; it shows [54 41 58] For string "LAPTOP" it shows [4C 41 50 54 4F 50] But when read that same parquet file from impala it has no issues. Can anyone suggest me a method to overcome this problem? Thank you, Best regards, Nipuna Shantha

RE: Encoded data retrieved when reading Parquet file

Solved this issue. Thank You On 2022/10/19 05:26:51 Nipuna Shantha wrote:> Hi all,> > I am writing to a parquet file using Impala version 3.4.0 and try to read> same parquet file from Spark 3.3.0 to a DataFrame.> > var df = spark.read.parquet(parquet_file_name)> > But w

Incremental Value dependents on another column of Data frame Spark

Hi all, This is the sample set of data that I used for this task [image: image.png] My expected output is as below [image: image.png] My scenario is if Type is M01 the count should be 0 and if Type is M02 it should be incremented from 1 or 0 until the sequence of M02 is finished. Imagine this

Dynamic value as the offset of lag() function

Hi all This is the sample set of data that I used for this task ,[image: image.png] My need is to pass count as the offset of lag() function. *[ lag(col(), lag(count)).over(windowspec) ]* But as the lag function expects lag(Column, Int) above code does not work. So can you guys suggest a