There is another feature missing for CalendarInterval, which is related
to comparability: measure the length of an interval.
Would be nice if you could access the length of an interval, than you
could compute something like this:
|Seq((Timestamp.valueOf("2020-02-01 12:00:00"),
Timestamp.valu
I compute the difference of two timestamps and compare them with a
constant interval:
Seq(("2019-01-02 12:00:00", "2019-01-02 13:30:00"))
.toDF("start", "end")
.select($"start".cast(TimestampType), $"end".cast(TimestampType))
.select($"start", $"end", ($"end" - $"start").as("diff"))
.whe
The problem is that there isn't a consistent number of seconds an interval
represents - as Wenchen mentioned, a month interval isn't a fixed number of
days. If your use case can account for that, maybe you could add the
interval to a fixed reference date and then compare the result.
On Tue, Feb 11
What's your use case to compare intervals? It's tricky in Spark as there is
only one interval type and you can't really compare one month with 30 days.
On Wed, Feb 12, 2020 at 12:01 AM Enrico Minack
wrote:
> Hi Devs,
>
> I would like to know what is the current roadmap of making
> CalendarInterv
Hi Devs,
I would like to know what is the current roadmap of making
CalendarInterval comparable and orderable again (SPARK-29679,
SPARK-29385, #26337).
With #27262, this got reverted but SPARK-30551 does not mention how to
go forward in this matter. I have found SPARK-28494, but this seems t
