Re: cast column string -> timestamp in Parquet file

The following snippet may help: sqlContext.read.parquet(path).withColumn("col_ts", $"col".cast(TimestampType)).drop("col") Cheng On 1/21/16 6:58 AM, Muthu Jayakumar wrote: DataFrame and udf. This may be more performant than doing an RDD transformation as you'll only transform just the

cast column string -> timestamp in Parquet file

Hi I have a large size parquet file . I need to cast the whole column to timestamp format , then save What the right way to do it ? Thanks a lot

Re: cast column string -> timestamp in Parquet file

DataFrame and udf. This may be more performant than doing an RDD transformation as you'll only transform just the column that requires to be changed. Hope this helps. On Thu, Jan 21, 2016 at 6:17 AM, Eli Super wrote: > Hi > > I have a large size parquet file . > > I need