Hi, i am trying to create dummy variables in sparkR by creating new columns for categorical variables. But it is not appending the columns
df <- createDataFrame(sqlContext, iris) class(dtypes(df)) cat.column<-vector(mode="character",length=nrow(df)) cat.column<-collect(select(df,df$Species)) lev<-length(levels(as.factor(unlist(cat.column)))) varb.names<-vector(mode="character",length=lev) for (i in 1:lev){ varb.names[i]<-paste0(colnames(cat.column),i) } for (j in 1:lev) { dummy.df.new<-withColumn(df,paste0(colnames (cat.column),j),if else(df$Species==levels(as.factor(un list(cat.column)) [j],1,0) ) } I am getting the below output for head(dummy.df.new) output: Sepal_Length Sepal_Width Petal_Length Petal_Width Species Species1 1 5.1 3.5 1.4 0.2 setosa 1 2 4.9 3.0 1.4 0.2 setosa 1 3 4.7 3.2 1.3 0.2 setosa 1 4 4.6 3.1 1.5 0.2 setosa 1 5 5.0 3.6 1.4 0.2 setosa 1 6 5.4 3.9 1.7 0.4 setosa 1 Problem: Species2 and Species3 column are not getting added to the dataframe -- Warm regards, Devesh.