Please unsubscribe me.
Thanks,
Alfredo
Thanks darling
I tried this and worked
hdfs getconf -confKey fs.defaultFS
hdfs://localhost:9000
scala> :paste
// Entering paste mode (ctrl-D to finish)
val textFile =
sc.textFile("hdfs://127.0.0.1:9000/hdfs/spark/examples/README.txt")
val counts = textFile.flatMap(line => line.split(" "))
Hi,
What is the unix_timestamp() function equivalent in a plain spark SQL query?
I want to subtract one timestamp column from another, but in plain SQL am
getting error "Should be numeric or calendarinterval and not timestamp."
But when I did through the above function inaide withColumn, it
I'm posting here so that if anyone else has similar problems, it might be
of help.
First problem:
I tried multiple different MySQL JDBC drivers to no avail, then I put Spark
into Uber-chatty mode (DEBUG) and looked at the Catalyst compiler and
started seeing datatime's being compared to NULL.
Hi,
Am unable to solve a comparison between two timestamp field's difference
and a particular interval of time in Spark SQL.
I've asked rhe question here: https://stackoverflow.com/questions/60995744
Thanks,
Aakash.
Once parsed into a Timestamp the timestamp is store internally as UTC
and printed as your local timezone (e.g. as defined by
spark.sql.session.timeZone). Spark is good at hiding timezone
information from you.
You can get the timezone information via date_format(column, format):
import
