[jira] [Created] (FLINK-11223) Time zone problem of row time field
sunjincheng created FLINK-11223:
---
Summary: Time zone problem of row time field
Key: FLINK-11223
URL: https://issues.apache.org/jira/browse/FLINK-11223
Project: Flink
Issue Type: Bug
Components: Table API & SQL
Affects Versions: 1.7.2, 1.8.0
Reporter: sunjincheng
SQL:
{code:java}
Data.+=(("001", "flink", new Timestamp(1545803501000L)))
tab...toTable(tEnv, 'a, 'b, 'rowtime2, 'rowtime.rowtime)
val sqlQuery = "SELECT rowtime2, rowtime FROM tab"{code}
Result:
{code:java}
2018-12-26 13:51:41.0, 2018-12-26 05:51:41.0
{code}
My local time zone is UTC/GMT+08.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (FLINK-11222) Change api.scala.DataStream to api.datastream.DataStream in HarnessTestBase
Hequn Cheng created FLINK-11222:
---
Summary: Change api.scala.DataStream to api.datastream.DataStream
in HarnessTestBase
Key: FLINK-11222
URL: https://issues.apache.org/jira/browse/FLINK-11222
Project: Flink
Issue Type: Improvement
Components: Table API & SQL
Reporter: Hequn Cheng
Thanks to FLINK-11074, we can create harness tester from a DataStream which
makes easier to write harness test.
However, it would be better if we change the parameter type from
api.scala.DataStream to api.datastream.DataStream for the
\{{createHarnessTester()}} method, so that both java.DataStream and
scala.DataStream can use this method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: [ANNOUNCE] Apache Flink 1.5.6 released
Thanks Thomas. It's nice to have a more stable flink 1.5.x vino yang 于2018年12月27日周四 上午9:43写道： > Thomas, thanks for being a release manager. > And Thanks for the whole community. > I think the release of Flink 1.5.6 makes sense for many users who are > currently unable to upgrade major versions. > > Best, > Vino > > jincheng sun 于2018年12月27日周四 上午8:00写道： > >> Thanks a lot for being our release manager Thomas. >> Thanks a lot for made this release possible! >> >> Cheers, >> Jincheng >> >> Thomas Weise 于2018年12月27日周四 上午4:03写道： >> >>> The Apache Flink community is very happy to announce the release of >>> Apache >>> Flink 1.5.6, which is the final bugfix release for the Apache Flink 1.5 >>> series. >>> >>> Apache Flink® is an open-source stream processing framework for >>> distributed, high-performing, always-available, and accurate data >>> streaming >>> applications. >>> >>> The release is available for download at: >>> https://flink.apache.org/downloads.html >>> >>> Please check out the release blog post for an overview of the >>> improvements >>> for this bugfix release: >>> https://flink.apache.org/news/2018/12/22/release-1.5.6.html >>> >>> The full release notes are available in Jira: >>> >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12344315 >>> >>> We would like to thank all contributors of the Apache Flink community who >>> made this release possible! >>> >>> Regards, >>> Thomas >>> >> -- Best Regards Jeff Zhang
Re: [ANNOUNCE] Apache Flink 1.5.6 released
Thomas, thanks for being a release manager. And Thanks for the whole community. I think the release of Flink 1.5.6 makes sense for many users who are currently unable to upgrade major versions. Best, Vino jincheng sun 于2018年12月27日周四 上午8:00写道： > Thanks a lot for being our release manager Thomas. > Thanks a lot for made this release possible! > > Cheers, > Jincheng > > Thomas Weise 于2018年12月27日周四 上午4:03写道： > >> The Apache Flink community is very happy to announce the release of Apache >> Flink 1.5.6, which is the final bugfix release for the Apache Flink 1.5 >> series. >> >> Apache Flink® is an open-source stream processing framework for >> distributed, high-performing, always-available, and accurate data >> streaming >> applications. >> >> The release is available for download at: >> https://flink.apache.org/downloads.html >> >> Please check out the release blog post for an overview of the improvements >> for this bugfix release: >> https://flink.apache.org/news/2018/12/22/release-1.5.6.html >> >> The full release notes are available in Jira: >> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12344315 >> >> We would like to thank all contributors of the Apache Flink community who >> made this release possible! >> >> Regards, >> Thomas >> >
Re: [DISCUSS] Enhance convenience of TableEnvironment in TableAPI/SQL
Hi Timo, Thanks for your summary of design in FLINK-11067's discuss！ This proposal has two core objectives which I mentioned: 1. Must solve user import problems； 2. Unify interface definitions of TableEnvironment for stream and batch. I think FLINK-11067 can fit the #1. and we need create a new JIRA. for #2. i.e. User only take one TableEnvironment, e.g.: ExecutionEnvironment env = … TableEnvironment tEnv = TableEnvironment.getTableEnvironment(env) Of course, stream and batch unification will undermine existing compatibility, but I still think this is an effort we must make. At the SQL/TableAPI level, users don't need to know the existence of `BatchTableEnvironment` and `StreamTableEnvironment`. What do you think？ Thanks, Jincheng jincheng sun 于2018年12月12日周三 下午3:44写道： > HI Timo, > > Thanks for your feedback! And I'm glad to hear that you are already > thinking about import issues! > > 1. I commented on the solution you mentioned in FLINK-11067. I have the > same questions with Dian Fu, about the design of compatibility in the > google doc, I look forward to your reply. > > 2. About unified stream batch interface definition > >> However, I don't like the design of putting all methods of Batch and >> Stream environments into the base class and throw exceptions if not >> supported by base classes. This sounds not like a nice object oriented >> design and confuses users. > > > At present, we have unified the stream and batch interface definitions on > the Table, such as the `orderBy` operator. Although it only supports time > order on the stream, we still have the interface definition unified, check > it at runtime, if you want `orderBy` string on the stream, will throw an > exception. So we should unify the interface definition of > TableEnvironment in some way. When the stream and batch execution modes are > unified and Stream/BatchSource/sink are unified , a job can be run in > mix(Stream/Batch) mode. By then, a table can be either a toDataSet or a > ToDataStream. > > 3. About Scala dependencies > IMO. It is not expected to solve the Scala dependency problem in this > proposal(keep status quo). The Scala dependency problem is the goal of > FLIP-28. > This proposal has two core objectives: > 1) Must solve user import problems； > 2) Do our best to unify interface definitions of TableEnvironment for > stream and batch. > > So, I think we can solve the user problem and unify the interface between > Stream and Batch firstly. Regarding the separation of Scala and Java, I > agree that when we do FLIP-28, we can have a Java abstraction and a Scala > abstraction in `flink-table-api.java` and `flink-table-api.scala` > respectively which we mentioned in Q/A session of google doc. > > Best, > Jincheng > > > Timo Walther 于2018年12月11日周二 下午3:13写道： > >> Hi Jincheng, >> >> thanks for the proposal. I totally agree with the problem of having 3 >> StreamTableEnvironments and 3 BatchTableEnvironments. We also identified >> this problem when doing Flink trainings and introductions to the Table & >> SQL API. >> >> Actually, @Dawid and I were already discussing to remove this >> shortcoming while working on FLINK-11067 [1]. The porting allows to fix >> the class hierarchy because some visibilities of members change as well >> from Scala to Java. This would not break backwards compatibility as the >> base classes should not be used by users anyway. >> >> However, I don't like the design of putting all methods of Batch and >> Stream environments into the base class and throw exceptions if not >> supported by base classes. This sounds not like a nice object oriented >> design and confuses users. >> >> I added some comments to the document. I think we can improve the >> current situation without breaking backwards compatibility. Methods that >> interact with Scala and Java API such as toDataSet/toDataStream should >> not be moved to an abstract class as they would otherwise pull in Scala >> dependencies transitively or do not incoperate with the type extraction >> logic of the target API. >> >> Regards, >> Timo >> >> [1] https://issues.apache.org/jira/browse/FLINK-11067 >> >> >> Am 11.12.18 um 06:12 schrieb Zhang, Xuefu: >> > Hi Jincheng, >> > >> > Thanks for bringing this up. It seems making good sense to me. However, >> one concern I have is about backward compatibility. Could you clarify >> whether existing user program will break with the proposed changes? >> > >> > The answer to the question would largely determine when this can be >> introduced. >> > >> > Thanks, >> > Xuefu >> > >> > >> > -- >> > Sender:jincheng sun >> > Sent at:2018 Dec 10 (Mon) 18:14 >> > Recipient:dev >> > Subject:[DISCUSS] Enhance convenience of TableEnvironment in >> TableAPI/SQL >> > >> > Hi All, >> > >> > According to the feedback from users, the design of TableEnvironment is >> very inconvenient for users, and often mistakenly imported by IDE, >> especially for Jav
Re: [RESULT] [VOTE] Release 1.7.1, release candidate #2
Looks like the Maven artifacts are not promoted yet: https://repo.maven.apache.org/maven2/org/apache/flink/flink-metrics-core/ https://repository.apache.org/content/repositories/orgapacheflink-1198/org/apache/flink/flink-metrics-core/ Thomas On Fri, Dec 21, 2018 at 9:29 AM Chesnay Schepler wrote: > |I'm happy to announce that we have unanimously approved this release.| > |There are 5 approving votes, 3 of which are binding: > * Hequn (non-binding) > * Timo (binding) > * Till (binding) > * jincheng (non-binding) > * Chesnay (binding)| > |There are no disapproving votes.| > |Thanks everyone!| >
Re: [ANNOUNCE] Apache Flink 1.5.6 released
Thanks a lot for being our release manager Thomas. Thanks a lot for made this release possible! Cheers, Jincheng Thomas Weise 于2018年12月27日周四 上午4:03写道： > The Apache Flink community is very happy to announce the release of Apache > Flink 1.5.6, which is the final bugfix release for the Apache Flink 1.5 > series. > > Apache Flink® is an open-source stream processing framework for > distributed, high-performing, always-available, and accurate data streaming > applications. > > The release is available for download at: > https://flink.apache.org/downloads.html > > Please check out the release blog post for an overview of the improvements > for this bugfix release: > https://flink.apache.org/news/2018/12/22/release-1.5.6.html > > The full release notes are available in Jira: > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12344315 > > We would like to thank all contributors of the Apache Flink community who > made this release possible! > > Regards, > Thomas >
[ANNOUNCE] Apache Flink 1.5.6 released
The Apache Flink community is very happy to announce the release of Apache Flink 1.5.6, which is the final bugfix release for the Apache Flink 1.5 series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming applications. The release is available for download at: https://flink.apache.org/downloads.html Please check out the release blog post for an overview of the improvements for this bugfix release: https://flink.apache.org/news/2018/12/22/release-1.5.6.html The full release notes are available in Jira: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12344315 We would like to thank all contributors of the Apache Flink community who made this release possible! Regards, Thomas
[jira] [Created] (FLINK-11221) Support delimited identifiers in TableAPI
Hequn Cheng created FLINK-11221:
---
Summary: Support delimited identifiers in TableAPI
Key: FLINK-11221
URL: https://issues.apache.org/jira/browse/FLINK-11221
Project: Flink
Issue Type: Improvement
Components: Table API & SQL
Reporter: Hequn Cheng
Assignee: Hequn Cheng
There are two kinds of identifiers in SQL: regular identifiers and delimited
identifiers. Different from regular identifiers, delimited identifiers are
identifiers enclosed in double backticks(``), double quotation marks (") or
brackets ([ ]).
Currently, delimited identifiers in double backticks have been supported in
Flink SQL(thanks to Calcite). In Calcite, delimited identifiers may contain
virtually any character, including spaces and other punctuation. With delimited
identifiers, we can name some special identifiers such as `a-a` which may be an
essential requirement.
However, delimited identifiers are not supported in TableApi. It would be nice
if we support it in TableApi.
Any suggestions are welcomed! Thank you.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (FLINK-11220) Can not Select row time field in JOIN query
sunjincheng created FLINK-11220:
---
Summary: Can not Select row time field in JOIN query
Key: FLINK-11220
URL: https://issues.apache.org/jira/browse/FLINK-11220
Project: Flink
Issue Type: Bug
Components: Table API & SQL
Affects Versions: 1.8.0
Reporter: sunjincheng
SQL:
{code:java}
Orders...toTable(tEnv, 'orderId, 'orderTime.rowtime)
Payment...toTable(tEnv, 'orderId, 'payTime.rowtime)
SELECT orderTime, o.orderId, payTime
FROM Orders AS o JOIN Payment AS p
ON o.orderId = p.orderId AND
p.payTime BETWEEN orderTime AND orderTime + INTERVAL '1' HOUR
{code}
Execption:
{code:java}
org.apache.flink.table.api.TableException: Found more than one rowtime field:
[orderTime, payTime] in the table that should be converted to a DataStream.
Please select the rowtime field that should be used as event-time timestamp for
the DataStream by casting all other fields to TIMESTAMP.
at
org.apache.flink.table.api.StreamTableEnvironment.translate(StreamTableEnvironment.scala:906)
{code}
The reason for the error is that we have 2 time fields `orderTime` and
`payTime`. I think we do not need throw the exception, and we can remove the
logic of `plan.process(new OutputRowtimeProcessFunction[A](conversion,
rowtimeFields.head.getIndex))`, if we want using the timestamp after
toDataSteram, we should using `assignTimestampsAndWatermarks()`.
What do you think ? [~twalthr] [~fhueske]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
