can't apply mappartitions to dataframe generated from carboncontext

[sunerhan1...@sina.com]

hi:
    appendix are the full error message
    I try to modify dataframe row in non-sql way and get Task not 
serializable,my test procedure like follows:
            1.val df=cc.sql(select * from t1)
            2.def function1 (iterator: Iterator[Row]):Iterator[Row]={
                    var list=scala.collection.mutable.ListBuffer[Row]()
                    while (iterator.hasNext) {
        var r=iterator.next
                        
if(r.getAs[String]("col1").toString.equalsIgnoreCase(r.getAs[String]("col2").toString))
 list+=r}
                        list.iterator
                }
            3.df.mapPartitions(r=>function1 (r))    
   I also apply function1 to dataframe generated from sqlContext work fine,
    so i believe carboncontext is refering some outer variables that are not 
serializable.

    



sunerhan1...@sina.com