Hi, As I cant add colmns from another Dataframe I am planning to my row coulmns to map of key and arrays As I am new to scala and spark I am trying like below
// create an empty map import scala.collection.mutable.{ArrayBuffer => mArrayBuffer} var map = Map[Int,mArrayBuffer[Any]]() def addNode(key: String, value:ArrayBuffer[Any] ) ={ nodes += (key -> (value :: (nodes get key getOrElse Nil))) } var rows = dfLnItmMappng.collect() rows.foreach(r => addNode(r.getInt(2), (r.getString(1),r.getString(3),r.getString(4),r.getString(5)))) for ((k,v) <- rows) printf("key: %s, value: %s\n", k, v) But I am getting below error : import scala.collection.mutable.{ArrayBuffer=>mArrayBuffer} map: scala.collection.immutable.Map[Int,scala.collection.mutable.ArrayBuffer[Any]] = Map() <console>:28: error: not found: value nodes nodes += (key -> (value :: (nodes get key getOrElse Nil))) ^ <console>:27: error: not found: type ArrayBuffer def addNode(key: String, value:ArrayBuffer[Any] ) ={ If anybody knows better method to add coulmns from another dataframe,please help by letting me know . Thanks, Divya