hi,all:
   my spark version is 2.0
   I have a parquet file with one colum name url type is string,I wang get 
substring from the url and add it to the datafram:
   val df = spark.read.parquet("/parquetdata/weixin/page/month=201607")
   val df2 = df.withColumn("pa_bid",when($"url".isNull,col("url").substr(3, 5)))
   df2.select("pa_bid","url").show
   +------+--------------------+
|pa_bid|                 url|
+------+--------------------+
|  null|http://mp.weixin....|
|  null|http://mp.weixin....|
|  null|http://mp.weixin....|
|  null|http://mp.weixin....|
|  null|http://mp.weixin....|
|  null|http://mp.weixin....|
|  null|http://mp.weixin....|
|  null|http://mp.weixin....|

Why what I got is null?


2016-12-06


lk_spark 

Reply via email to