Hi
I am trying to run Spark 2.0.0 in the Microsoft Windows environment without
hadoop or hive. I am running it in the local mode i.e. cmd> spark-shell and can
run the shell. When I try to run the sample example
here<http://spark..apache.org/docs/latest/sql-programming-guide.html#inferring-the-schema-using-reflection>
provided in the documentation, the spark execution fails to create the data
frame at .toDF()
// Create an RDD of Person objects from a text file, convert it to a Dataframe
val peopleDF = spark.sparkContext
..textFile("examples/src/main/resources/people.txt")
..map(_.split(","))
..map(attributes => Person(attributes(0), attributes(1).trim.toInt))
..toDF()
I have also run and tried these steps separately (split and Person) but it
fails at .toDF()
It give the sequence of exceptions like:
HiveException: java.lang.RuntimeException:
Unable to instantiate org...SessionHiveMetaStoreClient
..
..
IllegalArgumentException: java.net.URISyntaxException: Relative path in the
absolute URI
I have also defined the case class Person(age:String, age:Int)
I had successfully run codes for spark 1.4 on Windows. Is there any guide
available to configure spark 2.0.0 on Windows for above mentioned environment ?
Or it does not support ? Your inputs will be appreciated.
Source:
http://stackoverflow.com/questions/39538544/apache-spark-2-0-0-on-microsoft-windows-create-dataframe
Similar Post:
http://stackoverflow.com/questions/39402145/spark-windows-error-when-executing-todf
Regards
Advait Mohan Raut
Essex Lake Group LLC |
Mobile: +91-99-101-700-69<tel:+919910170069> |
Email: adv...@essexlg.com<mailto:adv...@essexlg.com> |
Skype: advait.raut
The information transmitted herewith is sensitive information intended only for
use to the individual or entity to which it is addressed. If the reader of this
message is not the intended recipient, you are hereby notified that any review,
retransmission, dissemination, distribution, copying or other use of, or taking
of any action in reliance upon, this information is strictly prohibited. If you
have received this communication in error, please contact the sender and delete
the material from your computer.
WARNING: E-mail communications cannot be guaranteed to be timely, secure,
error-free or virus-free. The recipient of this communication should check this
e-mail and each attachment for the presence of viruses. The sender does not
accept any liability for any errors or omissions in the content of this
electronic communication which arises as a result of e-mail transmission.
