Hi, Using concat is one of the way. But the + is more intuitive and easy to understand.
1427357...@qq.com From: Shmuel Blitz Date: 2018-03-26 15:31 To: 1427357...@qq.com CC: spark?users; dev Subject: Re: the issue about the + in column,can we support the string please? Hi, you can get the same with: import org.apache.spark.sql.functions._ import sqlContext.implicits._ import org.apache.spark.sql.types.{IntegerType, StringType, StructField, StructType} val schema = StructType(Array(StructField("name", StringType), StructField("age", IntegerType) )) val lst = List(Row("Shmuel", 13), Row("Blitz", 23)) val rdd = sc.parallelize(lst) val df = sqlContext.createDataFrame(rdd,schema) df.withColumn("newName", concat($"name" , lit("abc")) ).show() On Mon, Mar 26, 2018 at 6:36 AM, 1427357...@qq.com <1427357...@qq.com> wrote: Hi all, I have a table like below: +---+---------+-----------+ | id| name|sharding_id| +---+---------+-----------+ | 1|leader us| 10000| | 3| mycat| 10000| +---+---------+-----------+ My schema is : root |-- id: integer (nullable = false) |-- name: string (nullable = true) |-- sharding_id: integer (nullable = false) I want add a new column named newName. The new column is based on "name" and append "abc" after it. My code looks like: stud_scoreDF.withColumn("newName", stud_scoreDF.col("name") + "abc" ).show() When I run the code, I got the reslult: +---+---------+-----------+-------+ | id| name|sharding_id|newName| +---+---------+-----------+-------+ | 1|leader us| 10000| null| | 3| mycat| 10000| null| +---+---------+-----------+-------+ I checked the code, the key code is in arithmetic.scala. line 165. It looks like: override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = dataType match { case dt: DecimalType => defineCodeGen(ctx, ev, (eval1, eval2) => s"$eval1.$$plus($eval2)") case ByteType | ShortType => defineCodeGen(ctx, ev, (eval1, eval2) => s"(${ctx.javaType(dataType)})($eval1 $symbol $eval2)") case CalendarIntervalType => defineCodeGen(ctx, ev, (eval1, eval2) => s"$eval1.add($eval2)") case _ => defineCodeGen(ctx, ev, (eval1, eval2) => s"$eval1 $symbol $eval2") } My issue is: Can we add case StringType in this class to support string append please? 1427357...@qq.com -- Shmuel Blitz Big Data Developer Email: shmuel.bl...@similarweb.com www.similarweb.com