Thank you so much for the quick response and great help.
@jeff, I will use the library if the 3.1 release is getting delayed. Thank
you so much.
On Fri, Jan 29, 2021 at 1:23 PM Jeff Evans
wrote:
> If you need to do this in 2.x, this library does the trick:
> https://github.com/fqaiser94/mse
>
If you need to do this in 2.x, this library does the trick:
https://github.com/fqaiser94/mse
On Fri, Jan 29, 2021 at 12:15 PM Adam Binford wrote:
> I think they're voting on the next release candidate starting sometime
> next week. So hopefully barring any other major hurdles within the next
I think they're voting on the next release candidate starting sometime next
week. So hopefully barring any other major hurdles within the next few
weeks.
On Fri, Jan 29, 2021, 1:01 PM Felix Kizhakkel Jose <
felixkizhakkelj...@gmail.com> wrote:
> Wow, that's really great to know. Thank you so
Wow, that's really great to know. Thank you so much Adam. Do you know when
the 3.1 release is scheduled?
Regards,
Felix K Jose
On Fri, Jan 29, 2021 at 12:35 PM Adam Binford wrote:
> As of 3.0, the only way to do it is something that will recreate the whole
> struct:
>
As of 3.0, the only way to do it is something that will recreate the whole
struct:
df.withColumn('timingPeriod',
f.struct(f.col('timingPeriod.start').cast('timestamp').alias('start'),
f.col('timingPeriod.end').cast('timestamp').alias('end')))
There's a new method coming in 3.1 on the column class
Hello All,
I am using pyspark structured streaming and I am getting timestamp fields
as plain long (milliseconds), so I have to modify these fields into a
timestamp type
a sample json object object:
{
"id":{
"value": "f40b2e22-4003-4d90-afd3-557bc013b05e",
"type": "UUID",
Hello All,
I am using pyspark structured streaming and I am getting timestamp fields
as plain long (milliseconds), so I have to modify these fields into a
timestamp type
a sample json object object:
{
"id":{
"value": "f40b2e22-4003-4d90-afd3-557bc013b05e",
"type": "UUID",
Hi Sachit,
Yes it was executed using spark shell, history is already enabled. already
checked sql tab but it is not showing the query. My spark version is 2.4.5
Thanks,
Arpan Bhandari
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
Hi Arpan,
Was it executed using spark shell?
If yes type :history
Do u have history server enabled?
If yes , go to the history and go to the SQL tab in History UI.
Thanks
Sachit
On Fri, 29 Jan 2021, 19:19 Arpan Bhandari, wrote:
> Hi ,
>
> Is there a way to track back spark sql after it has
Hi ,
Is there a way to track back spark sql after it has been already run i.e.
query has been already submitted by a person and i have to back trace what
query actually got submitted.
Appreciate any help on this.
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
10 matches
Mail list logo
