Hi Hitesh,
Schema of the table is inferred automatically if you are reading from JSON
file, wherein when you are reading from a text file you will have to
provide a schema for the table you want to create (JSON has schema within
it).
You can create a data frames and register them as tables.
1.
Hi team,
I want to read the text file from s3. I am doing it using DataFrame. Like
below:-
DataFrame d=sql.read().text("s3://my_first_text_file.txt");
d.registerTempTable("table1");
DataFrame d1=sql.sql("Select * from table1");
d1.printSchema();
.n3.nabble.com/Directly-reading-data-from-S3-to-EC2-with-PySpark-tp24638p24698.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional
g py-spark shell?
>>
>> BR
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/Directly-reading-data-from-S3-to-EC2-with-PySpark-tp24638p24698.html
>> Sent from the Apache Spark User List mai
ning py-spark shell?
>
> BR
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Directly-reading-data-from-S3-to-EC2-with-PySpark-tp24638p24698.html
> Sent from the Apa
