unix timestamp with session timezone, will be
>> > convenient
>> > >>> for users, but will break the standard. I will +0.5 for this choice.
>> > >>>
>> > >>> yuxia 于2023年6月7日周三 12:06写道：
>> > >>>
>> &g
:00:00'`, we won't consider
> timezone.
> > >>>> But for the sql for timetravl:
> > >>>> `SELECT * FROM paimon_tb FOR SYSTEM_TIME AS OF TIMESTAMP '2023-04-27
> > >>>> 00:00:00'`, we will consider the timezone and convert to UTC
> > tim
s,
> >>>> Spark[1], Hive[2], Trino[3] also do the time conversion with
> >> considering
> >>>> the seesion time zone. I think we can align them to avoid the
> >>> inconsistency
> >>>> to other engines and provide convenie
>>> As for "violate the restriction outlined in FLINK-21978[1]", since we
>>> cast
>>>> timestamp to epochMillis only for the internal use, and won't expose it
>>> to
>>>> users, I don't think it will violate the restriction.
>>>> Bt
gBaseTable getTable(ObjectPath tablePath,
> > long
> > > timestamp)`. Maybe something like "timestamp of the table snapt, which
> is
> > > millseconds since 1970-01-01 00:00:00 UTC".
> > >
> > > [1]
> > >
> >
> https://github.com/
ala/org/apache/spark/sql/catalyst/analysis/TimeTravelSpec.scala#L56
> > [2]
> >
> https://github.com/apache/hive/blob/f5e69dc38d7ff26c70be19adc9d1a3ae90dc4cf2/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java#L989
> > [3]
> >
> https://github.com/trinodb/trino/blob/
t; https://github.com/trinodb/trino/blob/2433d9e60f1abb0d85c32374c1758525560e1a86/plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/IcebergMetadata.java#L443
>
>
> Best regards,
> Yuxia
>
> - 原始邮件 -
> 发件人: "Feng Jin"
> 收件人: "dev"
> 发送时间: 星期
plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/IcebergMetadata.java#L443
Best regards,
Yuxia
- 原始邮件 -
发件人: "Feng Jin"
收件人: "dev"
发送时间: 星期二, 2023年 6 月 06日 下午 10:15:47
主题: Re: [DISCUSS] FLIP-308: Support Time Travel In Batch Mode
Hi everyone
Thanks everyon
ckfill query between 2023-05-29
>> and 2023-06-04 in the past week, and the table schema changed on
>> 2023-06-01, will the query below detect the schema changes during backfill
>> the whole week?
>> >
>> > SELECT * FROM paimon_tb FOR SYSTEM_TIME AS OF
2023-06-01, will the query below detect the schema changes during backfill
> the whole week?
> >
> > SELECT * FROM paimon_tb FOR SYSTEM_TIME AS OF TIMESTAMP BETWEEN
> '2023-05-29 00:00:00' AND '2023-06-05 00:00:00'
> >
> > Best
> > Yun Tang
> >
> >
w detect the schema changes during backfill
> the whole week?
> >
> > SELECT * FROM paimon_tb FOR SYSTEM_TIME AS OF TIMESTAMP BETWEEN
> '2023-05-29 00:00:00' AND '2023-06-05 00:00:00'
> >
> > Best
> > Yun Tang
> >
> >
> > __
AS OF TIMESTAMP BETWEEN '2023-05-29
> 00:00:00' AND '2023-06-05 00:00:00'
>
> Best
> Yun Tang
>
>
>
> From: Shammon FY
> Sent: Thursday, June 1, 2023 17:57
> To: dev@flink.apache.org
> Subject: Re: [DISCUSS] FLIP-308: Support
0:00'
Best
Yun Tang
From: Shammon FY
Sent: Thursday, June 1, 2023 17:57
To: dev@flink.apache.org
Subject: Re: [DISCUSS] FLIP-308: Support Time Travel In Batch Mode
Hi Feng,
I have one minor comment about the public interface `Optional
getSnapshot()` in the `CatalogTable`.
As we can get
one question why the
> > >> implementation
> > >> of TimeTravel is delegated to Catalog? Assuming that we use Flink to
> > query
> > >> Hudi table with the time travel syntax, but we don't use the
> > HudiCatalog,
> > >> instead, we r
the rejected alternative should be considered.
> >>
> >> Best,
> >> Ron
> >>
> >> yuxia 于2023年5月30日周二 09:40写道：
> >>
> >> > Hi, Feng.
> >> > Notice this FLIP only support batch mode for time travel. Would it
> also
&g
;> > Notice this FLIP only support batch mode for time travel. Would it also
>> > make sense to support stream mode to a read a snapshot of the table as a
>> > bounded stream?
>> >
>> > Best regards,
>> > Yuxia
>> >
>> > - 原
table as a
> > bounded stream?
> >
> > Best regards,
> > Yuxia
> >
> > ----- 原始邮件 -
> > 发件人: "Benchao Li"
> > 收件人: "dev"
> > 发送时间: 星期一, 2023年 5 月 29日 下午 6:04:53
> > 主题: Re: [DISCUSS] FLIP-308: Support Time Travel I
quot;
> 发送时间: 星期一, 2023年 5 月 29日 下午 6:04:53
> 主题: Re: [DISCUSS] FLIP-308: Support Time Travel In Batch Mode
>
> # Can Calcite support this syntax ` VERSION AS OF` ?
>
> This also depends on whether this is defined in standard or any known
> databases that have implemented this. I
; > >
> > > >
> > > > Best,
> > > > Feng
> > > >
> > > >
> > > > On Thu, May 25, 2023 at 7:47 PM yuxia
> > > wrote:
> > > >
> > > > > Thanks Feng for bringing this up. It'll be
> > > Best,
> > > > Feng
> > > >
> > > >
> > > > On Thu, May 25, 2023 at 7:47 PM yuxia
> > > wrote:
> > > >
> > > > > Thanks Feng for bringing this up. It'll be great to introduce time
> > > travel
> > > > >
be great to introduce time
> > travel
> > > > to Flink to have a better integration with external data soruces.
> > > >
> > > > I also share same concern about the syntax.
> > > > I see in the part of `Whether to support other syntax
> implementati
support other syntax implementations`
> in
> > > this FLIP, seems the syntax in Calcite should be `FOR SYSTEM_TIME AS
> OF`,
> > > right?
> > > But the the syntax part in this FLIP, it seems to be `AS OF TIMESTAMP`
> > > instead of `FOR SYSTEM_TIME AS OF`. Is it
ms the syntax in Calcite should be `FOR SYSTEM_TIME AS OF`,
> > right?
> > But the the syntax part in this FLIP, it seems to be `AS OF TIMESTAMP`
> > instead of `FOR SYSTEM_TIME AS OF`. Is it just a mistake or by design?
> >
> >
> > Best regards,
> > Yuxi
ot;Benchao Li"
> 收件人: "dev"
> 发送时间: 星期四, 2023年 5 月 25日 下午 7:27:17
> 主题: Re: [DISCUSS] FLIP-308: Support Time Travel In Batch Mode
>
> Thanks Feng, it's exciting to have this ability.
>
> Regarding the syntax section, are you proposing `AS OF` instead of `
月 25日 下午 7:27:17
主题: Re: [DISCUSS] FLIP-308: Support Time Travel In Batch Mode
Thanks Feng, it's exciting to have this ability.
Regarding the syntax section, are you proposing `AS OF` instead of `FOR
SYSTEM AS OF` to do this? I know `FOR SYSTEM AS OF` is in the SQL standard
and has been suppor
Thanks Feng, it's exciting to have this ability.
Regarding the syntax section, are you proposing `AS OF` instead of `FOR
SYSTEM AS OF` to do this? I know `FOR SYSTEM AS OF` is in the SQL standard
and has been supported in some database vendors such as SQL Server. About
`AS OF`, is it in the
Also: How do we want to query the most recent version of a table?
`AS OF CURRENT_TIMESTAMP` would be ideal, but according to the docs both
the type is TIMESTAMP_LTZ and what is even more concerning is the it
actually is evalated row-based:
> Returns the current SQL timestamp in the local
Hi Feng,
thanks for proposing this FLIP. It makes a lot of sense to finally
support querying tables at a specific point in time or hopefully also
ranges soon. Following time-versioned tables.
Here is some feedback from my side:
1. Syntax
Can you elaborate a bit on the Calcite restrictions?
Hi, everyone.
I’d like to start a discussion about FLIP-308: Support Time Travel In Batch
Mode [1]
Time travel is a SQL syntax used to query historical versions of data. It
allows users to specify a point in time and retrieve the data and schema of
a table as it appeared at that time. With time
29 matches
Mail list logo
