Re: Re: Re: how to add colum to dataframe

2016-12-06 Thread lk_spark
QwOA==|http://mp.weixin| |MzAwOTIxMTcyMQ==|http://mp.weixin| |MzA3OTAyNzY2OQ==|http://mp.weixin| |MjM5NDAzMDAwMA==|http://mp.weixin| |MzAwMjE4MzU0Nw==|http://mp.weixin| |MzA4NzcyNjI0Mw==|http://mp.weixin| |MzI5OTE5Nzc5Ng==|http://mp.weixin| 2016-12-06 lk_spark

Re: Re: how to add colum to dataframe

2016-12-06 Thread lk_spark
thanks for reply. I will search how to use na.fill . and I don't know how to get the value of the column and do some operation like substr or split. 2016-12-06 lk_spark 发件人:Pankaj Wahane <pankajwah...@live.com> 发送时间:2016-12-06 17:39 主题:Re: how to add colum to dataframe 收件人:"lk_s

Re: how to add colum to dataframe

2016-12-06 Thread Pankaj Wahane
You may want to try using df2.na.fill(…) From: lk_spark <lk_sp...@163.com> Date: Tuesday, 6 December 2016 at 3:05 PM To: "user.spark" <user@spark.apache.org> Subject: how to add colum to dataframe hi,all: my spark version is 2.0 I have a parquet file with one colum n

how to add colum to dataframe

2016-12-06 Thread lk_spark
hi,all: my spark version is 2.0 I have a parquet file with one colum name url type is string,I wang get substring from the url and add it to the datafram: val df = spark.read.parquet("/parquetdata/weixin/page/month=201607") val df2 =