Re: sparkR not able to create /append new columns

browse/SPARK-12225) which is still under > discussion. If you desire this feature, you could comment on it. > > > > *From:* Franc Carter [mailto:franc.car...@gmail.com] > *Sent:* Wednesday, February 3, 2016 7:40 PM > *To:* Devesh Raj Singh > *Cc:* user@spark.apache.org > *Subj

sparkR not able to create /append new columns

Hi, i am trying to create dummy variables in sparkR by creating new columns for categorical variables. But it is not appending the columns df <- createDataFrame(sqlContext, iris) class(dtypes(df)) cat.column<-vector(mode="character",length=nrow(df)) cat.column<-collect(select(df,df$Species))

Re: sparkR not able to create /append new columns

Yes, I didn't work out how to solve that - sorry On 3 February 2016 at 22:37, Devesh Raj Singh wrote: > Hi, > > but "withColumn" will only add once, if i want to add columns to the same > dataframe in a loop it will keep overwriting the added column and in the > end the

Re: sparkR not able to create /append new columns

I had problems doing this as well - I ended up using 'withColumn', it's not particularly graceful but it worked (1.5.2 on AWS EMR) cheerd On 3 February 2016 at 22:06, Devesh Raj Singh wrote: > Hi, > > i am trying to create dummy variables in sparkR by creating new

Re: sparkR not able to create /append new columns

Hi, but "withColumn" will only add once, if i want to add columns to the same dataframe in a loop it will keep overwriting the added column and in the end the last added column( in the loop) will be the added column. like in my code above. On Wed, Feb 3, 2016 at 5:05 PM, Franc Carter

RE: sparkR not able to create /append new columns

/SPARK-12225) which is still under discussion. If you desire this feature, you could comment on it. From: Franc Carter [mailto:franc.car...@gmail.com] Sent: Wednesday, February 3, 2016 7:40 PM To: Devesh Raj Singh Cc: user@spark.apache.org Subject: Re: sparkR not able to create /append new columns