Hi Xi,
Yes,
You can do the following:
val sc = new SparkContext("local[2]", "mptest")
// or .. val sc = new SparkContext("spark://master:7070", "mptest")
val fileDataRdd = sc.textFile("/path/to/dir")
val fileLines = fileDataRdd.take(100)
The key here - i.e. the answer to your specific question - is the creation
of the SparkContext : that is one of the things spark-submit is doing for
you
2015-03-05 17:19 GMT-08:00 Xi Shen <davidshe...@gmail.com>:
> Hi,
>
> I am new to Spark. I see every spark program has a main() function. I
> wonder if I can run the spark program directly, without using spark-submit.
> I think it will be easier for early development and debug.
>
>
> Thanks,
> David
>
>
