Thank you very much Charles, I got it :)
On Sat, Jan 31, 2015 at 2:20 AM, Charles Feduke <charles.fed...@gmail.com>
wrote:
> You'll still need to:
>
> import org.apache.spark.SparkContext._
>
> Importing org.apache.spark._ does _not_ recurse into sub-objects or
> sub-packages, it only brings in whatever is at the level of the package or
> object imported.
>
> SparkContext._ has some implicits, one of them for adding groupByKey to an
> RDD[_] IIRC.
>
>
> On Fri Jan 30 2015 at 3:48:22 PM Stephen Boesch <java...@gmail.com> wrote:
>
>> Amit - IJ will not find it until you add the import as Sean mentioned.
>> It includes implicits that intellij will not know about otherwise.
>>
>> 2015-01-30 12:44 GMT-08:00 Amit Behera <amit.bd...@gmail.com>:
>>
>> I am sorry Sean.
>>>
>>> I am developing code in intelliJ Idea. so with the above dependencies I
>>> am not able to find *groupByKey* when I am searching by ctrl+<space>
>>>
>>>
>>> On Sat, Jan 31, 2015 at 2:04 AM, Sean Owen <so...@cloudera.com> wrote:
>>>
>>>> When you post a question anywhere, and say "it's not working", you
>>>> *really* need to say what that means.
>>>>
>>>>
>>>> On Fri, Jan 30, 2015 at 8:20 PM, Amit Behera <amit.bd...@gmail.com>
>>>> wrote:
>>>> > hi all,
>>>> >
>>>> > my sbt file is like this:
>>>> >
>>>> > name := "Spark"
>>>> >
>>>> > version := "1.0"
>>>> >
>>>> > scalaVersion := "2.10.4"
>>>> >
>>>> > libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0"
>>>> >
>>>> > libraryDependencies += "net.sf.opencsv" % "opencsv" % "2.3"
>>>> >
>>>> >
>>>> > code:
>>>> >
>>>> > object SparkJob
>>>> > {
>>>> >
>>>> > def pLines(lines:Iterator[String])={
>>>> > val parser=new CSVParser()
>>>> > lines.map(l=>{val vs=parser.parseLine(l)
>>>> > (vs(0),vs(1).toInt)})
>>>> > }
>>>> >
>>>> > def main(args: Array[String]) {
>>>> > val conf = new SparkConf().setAppName("Spark
>>>> Job").setMaster("local")
>>>> > val sc = new SparkContext(conf)
>>>> > val data = sc.textFile("/home/amit/testData.csv").cache()
>>>> > val result = data.mapPartitions(pLines).groupByKey
>>>> > //val list = result.filter(x=> {(x._1).contains("24050881")})
>>>> >
>>>> > }
>>>> >
>>>> > }
>>>> >
>>>> >
>>>> > Here groupByKey is not working . But same thing is working from
>>>> spark-shell.
>>>> >
>>>> > Please help me
>>>> >
>>>> >
>>>> > Thanks
>>>> >
>>>> > Amit
>>>>
>>>
>>>
