A few small-scale code style tips which might improve readability: You can in some cases write something like _.split("...") instead of x => x.split("...") A series of if-else conditions on x(15) can be replaced with "x(15) match { case"B" => ... }" .isEmpty may be more readable than == "" With so many fields, you may consider writing a simple case class to represent the object instead so you can write ".someField" instead of "(11)"
On Mon, Oct 12, 2015 at 12:28 PM, kali.tumm...@gmail.com <kali.tumm...@gmail.com> wrote: > Hi All, > > just wonderign is there any better way of writing this below code, I am new > to spark an I feel what I wrote is pretty simple and basic and straight > forward is there any better way of writing using functional paradigm. > > val QuoteRDD=quotefile.map(x => x.split("\\|")). > filter(line => line(0).contains("1017")). > map(x => ((x(5)+x(4)) , (x(5),x(4),x(1) , > if (x(15) =="B") > ( > {if (x(25) == "") x(9) else x(25)}, > {if (x(37) == "") x(11) else x(37)} > ) > else if (x(15) =="C" ) > ( > {if (x(24) == "") (x(9)) else x(24)}, > {if (x(30) == "") (x(11)) else x(30)} > ) > else if (x(15) =="A") > {(x(9),x(11))} > ))) > > Thanks > Sri > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Is-there-any-better-way-of-writing-this-code-tp25027.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org