Thanks.It works.
On Mon, Dec 5, 2016 at 2:03 PM, Michal Šenkýř wrote:
> Yet another approach:
> scala> val df1 = df.selectExpr("client_id",
> "from_unixtime(ts/1000,'-MM-dd')
> as ts")
>
> Mgr. Michal Šenkýřmike.sen...@gmail.com
> +420 605 071 818
>
> On 5.12.2016 09:22, Deepak Sharma wrote
Yet another approach:
scala> val df1 = df.selectExpr("client_id",
"from_unixtime(ts/1000,'-MM-dd') as ts")
Mgr. Michal Šenkýř
mike.sen...@gmail.com
+420 605 071 818
On 5.12.2016 09:22, Deepak Sharma wrote:
Another simpler approach will be:
scala> val findf = sqlContext.sql("select
clien
Another simpler approach will be:
scala> val findf = sqlContext.sql("select
client_id,from_unixtime(ts/1000,'-MM-dd') ts from ts")
findf: org.apache.spark.sql.DataFrame = [client_id: string, ts: string]
scala> findf.show
++--+
| client_id|ts|
+
This is the correct way to do it.The timestamp that you mentioned was not
correct:
scala> val ts1 = from_unixtime($"ts"/1000, "-MM-dd")
ts1: org.apache.spark.sql.Column = fromunixtime((ts / 1000),-MM-dd)
scala> val finaldf = df.withColumn("ts1",ts1)
finaldf: org.apache.spark.sql.DataFrame
This is how you can do it in scala:
scala> val ts1 = from_unixtime($"ts", "-MM-dd")
ts1: org.apache.spark.sql.Column = fromunixtime(ts,-MM-dd)
scala> val finaldf = df.withColumn("ts1",ts1)
finaldf: org.apache.spark.sql.DataFrame = [client_id: string, ts: string,
ts1: string]
scala> finald
Hi,
Thanks for replying to my question.
I am using scala
On Mon, Dec 5, 2016 at 1:20 PM, Marco Mistroni wrote:
> Hi
> In python you can use date time.fromtimestamp(..).
> strftime('%Y%m%d')
> Which spark API are you using?
> Kr
>
> On 5 Dec 2016 7:38 am, "Devi P.V" wrote:
>
>> Hi
Hi
In python you can use date
time.fromtimestamp(..).strftime('%Y%m%d')
Which spark API are you using?
Kr
On 5 Dec 2016 7:38 am, "Devi P.V" wrote:
> Hi all,
>
> I have a dataframe like following,
>
> ++---+
> |client_id
Hi all,
I have a dataframe like following,
++---+
|client_id |timestamp|
++---+
|cd646551-fceb-4166-acbc-b9|1477989416803 |
|3bc61951-0f49-43bf-9848-b2|1477983725292 |
|688a