Re: [ANNOUNCE] New Flink PMC member: Yang Wang

[Forward Xu]

Congratulations, Yang!


Best,

Forward

Jingsong Li  于2022年5月6日周五 10:07写道：

> Congratulations, Yang!
>
> Best,
> Jingsong
>
> On Fri, May 6, 2022 at 10:04 AM yuxia  wrote:
> >
> > Congratulations, Yang!
> >
> > Best regards,
> > Yuxia
> >
> > - 原始邮件 -
> > 发件人: "Zhu Zhu" 
> > 收件人: "dev" 
> > 抄送: "Yang Wang" 
> > 发送时间: 星期四, 2022年 5 月 05日 下午 10:36:19
> > 主题: Re: [ANNOUNCE] New Flink PMC member: Yang Wang
> >
> > Congratulations, Yang!
> >
> > Thanks,
> > Zhu
> >
> > Weiqing Yang  于2022年5月5日周四 22:28写道：
> > >
> > > Congratulations Yang!
> > >
> > > Best,
> > > Weiqing
> > >
> > > On Thu, May 5, 2022 at 4:18 AM Xintong Song 
> wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'm very happy to announce that Yang Wang has joined the Flink PMC!
> > > >
> > > > Yang has been consistently contributing to our community, by
> contributing
> > > > codes, participating in discussions, mentoring new contributors,
> answering
> > > > questions on mailing lists, and giving talks on Flink at
> > > > various conferences and events. He is one of the main contributors
> and
> > > > maintainers in Flink's Native Kubernetes / Yarn integrations and the
> Flink
> > > > Kubernetes Operator.
> > > >
> > > > Congratulations and welcome, Yang!
> > > >
> > > > Thank you~
> > > >
> > > > Xintong Song (On behalf of the Apache Flink PMC)
> > > >
>

Re: [Vote] FLIP-157 Migrate Flink Documentation from Jekyll to Hugo

[Forward Xu]

+1

Dian Fu  于2021年1月19日周二 上午11:40写道：

> +1
>
> > 在 2021年1月19日，上午11:34，Jark Wu  写道：
> >
> > +1
> >
> > On Tue, 19 Jan 2021 at 01:59, Till Rohrmann 
> wrote:
> >
> >> +1,
> >>
> >> Cheers,
> >> Till
> >>
> >> On Mon, Jan 18, 2021 at 4:12 PM Chesnay Schepler 
> >> wrote:
> >>
> >>> +1
> >>> On 1/18/2021 3:50 PM, Seth Wiesman wrote:
>  Addendum, 72 hours from now is Thursday the 21st :)
> 
>  sorry for the mistake.
> 
>  Seth
> 
>  On Mon, Jan 18, 2021 at 8:41 AM Timo Walther 
> >> wrote:
> 
> > +1
> >
> > Thanks for upgrading our docs infrastructure.
> >
> > Regards,
> > Timo
> >
> > On 18.01.21 15:29, Seth Wiesman wrote:
> >> Hi devs,
> >>
> >> The discussion of the FLIP-157 [1] seems has reached a consensus
> >>> through
> >> the mailing thread [2]. I would like to start a vote for it.
> >>
> >> The vote will be opened until 20th January (72h), unless there is an
> >> objection or no enough votes.
> >>
> >> Best,
> >> Seth
> >>
> >> [1]
> >>
> >
> >>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-157+Migrate+Flink+Documentation+from+Jekyll+to+Hugo
> >> [2]
> >>
> >
> >>>
> >>
> https://lists.apache.org/thread.html/r88152bf178381c5e3bc2d7b3554cea3d61cff9ac0edb713dc518d9c7%40%3Cdev.flink.apache.org%3E
> >
> >>>
> >>>
> >>
>
>

[jira] [Created] (FLINK-20587) INSERT INTO TIMESTAMP WITH LOCAL TIME ZONE VALUES, THROW Unsupported type

[Forward Xu (Jira)]

Forward Xu created FLINK-20587:
--

 Summary: INSERT INTO  TIMESTAMP WITH LOCAL TIME ZONE VALUES, THROW 
Unsupported type
 Key: FLINK-20587
 URL: https://issues.apache.org/jira/browse/FLINK-20587
 Project: Flink
  Issue Type: Improvement
  Components: Table SQL / Ecosystem
Affects Versions: 1.11.2, 1.11.1
Reporter: Forward Xu






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [ANNOUNCE] New Flink Committer: David Anderson

[Forward Xu]

Congrats David!


Best,

Forward

Peter Huang  于2020年8月13日周四 上午9:39写道：

> Congrats David!
>
> On Wed, Aug 12, 2020 at 6:37 PM godfrey he  wrote:
>
> > Congrats David!
> >
> > Best,
> > Godfrey
> >
> > Danny Chan  于2020年8月13日周四 上午8:57写道：
> >
> > > Congratulations, David!
> > >
> > > jincheng sun 于2020年8月13日 周四上午8:54写道：
> > >
> > > > Congrats and welcome David!
> > > >
> > > > Best,
> > > > Jincheng
> > > >
> > > >
> > > > David Anderson  于2020年8月13日周四 上午2:51写道：
> > > >
> > > > > Thank you one and all for the warm reception!
> > > > >
> > > > > I'm honored to join the group of committers, and look forward to
> > > > continuing
> > > > > to work with the community to improve and evangelize Flink.
> > > > >
> > > > > Best regards,
> > > > > David
> > > > >
> > > > > On Wed, Aug 12, 2020 at 5:08 PM Igal Shilman 
> > > wrote:
> > > > >
> > > > > > Congrats David!
> > > > > >
> > > > > > On Wed, Aug 12, 2020 at 4:18 PM Seth Wiesman <
> sjwies...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Congratulations David!
> > > > > > >
> > > > > > > Seth
> > > > > > >
> > > > > > > On Wed, Aug 12, 2020 at 8:47 AM Yuan Mei <
> yuanmei.w...@gmail.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Congrats, David!
> > > > > > > >
> > > > > > > > Best
> > > > > > > >
> > > > > > > > Yuan
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Aug 12, 2020 at 6:04 PM Rui Li <
> lirui.fu...@gmail.com>
> > > > > wrote:
> > > > > > > >
> > > > > > > > > Congratulations!
> > > > > > > > >
> > > > > > > > > On Wed, Aug 12, 2020 at 5:56 PM Xintong Song <
> > > > > tonysong...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Congratulations, David~!
> > > > > > > > > >
> > > > > > > > > > Thank you~
> > > > > > > > > >
> > > > > > > > > > Xintong Song
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Wed, Aug 12, 2020 at 5:11 PM Zhijiang <
> > > > > > wangzhijiang...@aliyun.com
> > > > > > > > > > .invalid>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Congrats, David!
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > >
> --
> > > > > > > > > > > From:Zhu Zhu 
> > > > > > > > > > > Send Time:2020年8月12日(星期三) 17:08
> > > > > > > > > > > To:dev 
> > > > > > > > > > > Subject:Re: [ANNOUNCE] New Flink Committer: David
> > Anderson
> > > > > > > > > > >
> > > > > > > > > > > Congratulations, David!
> > > > > > > > > > >
> > > > > > > > > > > Paul Lam  于2020年8月12日周三
> 下午4:49写道：
> > > > > > > > > > >
> > > > > > > > > > > > Congrats, David!
> > > > > > > > > > > >
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Paul Lam
> > > > > > > > > > > >
> > > > > > > > > > > > > 2020年8月12日 16:46，Benchao Li 
> > 写道：
> > > > > > > > > > > > >
> > > > > > > > > > > > > Congratulations!  David
> > > > > > > > > > > > >
> > > > > > > > > > > > > Leonard Xu  于2020年8月12日周三
> > 下午4:01写道：
> > > > > > > > > > > > >
> > > > > > > > > > > > >> Congratulations!  David
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Best
> > > > > > > > > > > > >> Leonard
> > > > > > > > > > > > >>> 在 2020年8月12日，15:59，Till Rohrmann <
> > > trohrm...@apache.org
> > > > >
> > > > > > 写道：
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> Congratulations, David!
> > > > > > > > > > > > >>
> > > > > > > > > > > > >>
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > >
> > > > > > > > > > > > > Best,
> > > > > > > > > > > > > Benchao Li
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best regards!
> > > > > > > > > Rui Li
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New PMC member: Piotr Nowojski

[Forward Xu]

Congratulations!


Best，

Forward

godfrey he  于2020年7月22日周三 上午10:45写道：

> Congratulations!
>
> Best,
> Godfrey
>
> Till Rohrmann  于2020年7月21日周二 下午10:46写道：
>
> > Congrats, Piotr!
> >
> > Cheers,
> > Till
> >
> > On Thu, Jul 9, 2020 at 4:15 AM Xingcan Cui  wrote:
> >
> > > Congratulations, Piotr!
> > >
> > > Best, Xingcan
> > >
> > > On Wed, Jul 8, 2020, 21:53 Yang Wang  wrote:
> > >
> > > > Congratulations Piotr!
> > > >
> > > >
> > > > Best,
> > > > Yang
> > > >
> > > > Dan Zou  于2020年7月8日周三 下午10:36写道：
> > > >
> > > > > Congratulations!
> > > > >
> > > > > Best,
> > > > > Dan Zou
> > > > >
> > > > > > 2020年7月8日 下午5:25，godfrey he  写道：
> > > > > >
> > > > > > Congratulations
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New PMC member: Piotr Nowojski

[Forward Xu]

Congratulations Piotr!


Best,

Forward

Matt Wang  于2020年7月7日周二 上午10:02写道：

> Congratulations!
>
>
> --
>
> Best,
> Matt Wang
>
>
> On 07/7/2020 09:45，Xingbo Huang wrote：
> Congratulations Piotr!
>
> Best,
> Xingbo
>
> Dian Fu  于2020年7月7日周二 上午9:43写道：
>
> Congrats, Piotr!
>
> 在 2020年7月7日，上午7:06，Jeff Zhang  写道：
>
> Congratulations Piotr!
>
> Thomas Weise  于2020年7月7日周二 上午4:45写道：
>
> Congratulations!
>
>
> On Mon, Jul 6, 2020 at 10:58 AM Seth Wiesman 
> wrote:
>
> Congratulations Piotr!
>
> On Mon, Jul 6, 2020 at 12:53 PM Peter Huang <
> huangzhenqiu0...@gmail.com>
> wrote:
>
> Congratulations!
>
> On Mon, Jul 6, 2020 at 10:36 AM Arvid Heise 
> wrote:
>
> Congratulations!
>
> On Mon, Jul 6, 2020 at 7:07 PM Stephan Ewen 
> wrote:
>
> Hi all!
>
> It is my pleasure to announce that Piotr Nowojski joined the Flink
> PMC.
>
> Many of you may know Piotr from the work he does on the data
> processing
> runtime and the network stack, from the mailing list, or the
> release
> manager work.
>
> Congrats, Piotr!
>
> Best,
> Stephan
>
>
>
> --
>
> Arvid Heise | Senior Java Developer
>
> 
>
> Follow us @VervericaData
>
> --
>
> Join Flink Forward  - The Apache Flink
> Conference
>
> Stream Processing | Event Driven | Real Time
>
> --
>
> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
>
> --
> Ververica GmbH
> Registered at Amtsgericht Charlottenburg: HRB 158244 B
> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason,
> Ji
> (Toni) Cheng
>
>
>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
>

Re: [ANNOUNCE] Yu Li is now part of the Flink PMC

[Forward Xu]

Congratulations Yu!


Best,

Forward

Jark Wu  于2020年6月17日周三 上午10:36写道：

> Congratulations Yu! Well deserved!
>
> Best,
> Jark
>
> On Wed, 17 Jun 2020 at 10:18, Haibo Sun  wrote:
>
> > Congratulations Yu!
> >
> > Best,
> > Haibo
> >
> >
> > At 2020-06-17 09:15:02, "jincheng sun"  wrote:
> > >Hi all,
> > >
> > >On behalf of the Flink PMC, I'm happy to announce that Yu Li is now
> > >part of the Apache Flink Project Management Committee (PMC).
> > >
> > >Yu Li has been very active on Flink's Statebackend component, working on
> > >various improvements, for example the RocksDB memory management for
> 1.10.
> > >and keeps checking and voting for our releases, and also has
> successfully
> > >produced two releases(1.10.0&1.10.1) as RM.
> > >
> > >Congratulations & Welcome Yu Li!
> > >
> > >Best,
> > >Jincheng (on behalf of the Flink PMC)
> >
> >
>

Re: [ANNOUNCE] New Flink Committer: Benchao Li

[Forward Xu]

Congratulations, Benchao

Best,
Forward

Jark Wu  于2020年6月9日周二 下午2:10写道：

> Hi everyone,
>
> On behalf of the PMC, I'm very happy to announce Benchao Li as a new Apache
> Flink committer.
>
> Benchao started contributing to Flink since late 2018. He is very active in
> Flink SQL component,
> and has also participated in many discussions, bug fixes. Over the past few
> months, he helped tremendously in answering user questions in the mailing
> list.
>
> Please join me in congratulating Benchao for becoming a Flink committer!
>
> Thanks,
> Jark
>

Re: [Discussion] flink elasticsearch connector supports

[Forward Xu]

hi Jacky Lau :

I agree with jark's point of view, the use of es is not just to read data,
more use is to group query, aggregate these.

Best,
Forward

Jacky Lau  于2020年6月5日周五 下午2:47写道：

> hi Etienne Chauchot:
> you can read here https://www.jianshu.com/p/d32e17dab90c, which is
> chinese.But you can konw that slice api has poor performance in es-hadoop
> project .
>
> And i found that es-hadoop has removed this and disable sliced scrolls by
> default. you can see below, which i found in the lastest es-hadoop release
> version
>  Configuration Changes
> `es.input.use.sliced.partitions` is deprecated in 6.5.0, and will be
> removed
> in 7.0.0. The default value for `es.input.max.docs.per.partition` (10)
> will also be removed in 7.0.0, thus disabling sliced scrolls by default,
> and
> switching them to be an explicitly opt-in feature.
>
> added[5.0.0]
> `es.input.max.docs.per.partition` ::
> When reading from an {es} cluster that supports scroll slicing ({es} v5.0.0
> and above), this parameter advises the
> connector on what the maximum number of documents per input partition
> should
> be. The connector will sample and estimate
> the number of documents on each shard to be read and divides each shard
> into
> input slices using the value supplied by
> this property. This property is a suggestion, not a guarantee. The final
> number of documents per partition is not
> guaranteed to be below this number, but rather, they will be close to this
> number. This property is ignored if you are
> reading from an {es} cluster that does not support scroll slicing ({es} any
> version below v5.0.0). By default, this
> value is unset, and the input partitions are calculated based on the number
> of shards in the indices being read.
>
>
>
> Jacky Lau wrote
> > hi Etienne Chauchot:
> > thanks for your discussion.
> > for 1) we do not supprt es  unbouded source currently
> >
> > for 2) RichParallelSourceFunction is used for streaming ,InputFormat is
> > for
> > batch
> >
> > for 3)  i downloaded beam just now. and the beam es connector is also
> > using
> > es-hadoop. i have read the code of es-hadoop(inputsplit contains shard
> and
> > slice. And i think it is better when diffirent shard has diffirent number
> > of
> > docs), which you can seed here
> > .https://github.com/elastic/elasticsearch-hadoop. But the code is not
> > good.
> > so we do not want to reference . and you can see presto, there is also
> > just
> > using inputsplit with shard not contains slice
> >
> > for 4) because flink es connectro has alreay using diffrent client (es 5
> > for
> > tranport client, es 6,7 for highlevelrest), we just  reuse it,which will
> > not
> > change too much code
> >
> >
> >
> > --
> > Sent from:
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>
>
>
>
>
> --
> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>

Re: [ANNOUNCE] New Apache Flink Committer - Xintong Song

[Forward Xu]

Congratulations Xintong


Best,

Forward

Marta Paes Moreira  于2020年6月5日周五 下午2:03写道：

> Congratulations, Tony! Great job!
>
> On Fri, Jun 5, 2020 at 8:00 AM Yang Wang  wrote:
>
> > Congratulations, Xintong.
> > Well deserved.
> >
> > Best,
> > Yang
> >
> > Weihua Hu  于2020年6月5日周五 下午1:57写道：
> >
> > > Congratulations!
> > >
> > > Best
> > > Weihua Hu
> > >
> > > > 2020年6月5日 13:50，Xingbo Huang  写道：
> > > >
> > > > Congratulations, Xintong.
> > > >
> > > > Best,
> > > > Xingbo
> > > >
> > > > Jiayi Liao  于2020年6月5日周五 下午1:49写道：
> > > >
> > > >> Congratulations!
> > > >>
> > > >> Best,
> > > >> Jiayi Liao
> > > >>
> > > >> On Fri, Jun 5, 2020 at 1:48 PM Biao Liu  wrote:
> > > >>
> > > >>> Congrats!
> > > >>>
> > > >>> Thanks,
> > > >>> Biao /'bɪ.aʊ/
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Fri, 5 Jun 2020 at 13:32, Thomas Weise  wrote:
> > > >>>
> > >  Congratulations!
> > > 
> > > 
> > >  On Thu, Jun 4, 2020, 10:17 PM Yuan Mei 
> > > wrote:
> > > 
> > > > Congrats, Xintong!
> > > >
> > > > On Fri, Jun 5, 2020 at 12:45 PM Becket Qin  >
> > > >>> wrote:
> > > >
> > > >> Hi all,
> > > >>
> > > >> On behalf of the PMC, I’m very happy to announce Xintong Song
> as a
> > > >>> new
> > > >> Flink committer.
> > > >>
> > > >> Xintong started to contribute to Flink about two years ago and
> has
> > > >>> been
> > > >> active since. His major work is in Flink resource management,
> and
> > > >>> have
> > > > also
> > > >> participated in discussions, bug fixes and answering questions.
> > > >>
> > > >> Please join me in congratulating Xintong for becoming a Flink
> > >  committer!
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Jiangjie (Becket) Qin
> > > >>
> > > >
> > > 
> > > >>>
> > > >>
> > >
> > >
> >
>

Re: [DISCUSS] Hierarchies in ConfigOption

[Forward Xu]

Here I have a little doubt. At present, our json only supports the
conventional json format. If we need to implement json with bson, json with
avro, etc., how should we express it?
Do you need like the following:

‘format.name' = 'json',

‘format.json.fail-on-missing-field' = 'false'


‘format.name' = 'bson',

‘format.bson.fail-on-missing-field' = ‘false'


Best,

Forward

Benchao Li  于2020年4月30日周四 上午9:58写道：

> Thanks Timo for staring the discussion.
>
> Generally I like the idea to keep the config align with a standard like
> json/yaml.
>
> From the user's perspective, I don't use table configs from a config file
> like yaml or json for now,
> And it's ok to change it to yaml like style. Actually we didn't know that
> this could be a yaml like
> configuration hierarchy. If it has a hierarchy, we maybe consider that in
> the future to load the
> config from a yaml/json file.
>
> Regarding the name,
> 'format.kind' looks fine to me. However there is another name from the top
> of my head:
> 'format.name', WDYT?
>
> Dawid Wysakowicz  于2020年4月29日周三 下午11:56写道：
>
> > Hi all,
> >
> > I also wanted to share my opinion.
> >
> > When talking about a ConfigOption hierarchy we use for configuring Flink
> > cluster I would be a strong advocate for keeping a yaml/hocon/json/...
> > compatible style. Those options are primarily read from a file and thus
> > should at least try to follow common practices for nested formats if we
> > ever decide to switch to one.
> >
> > Here the question is about the properties we use in SQL statements. The
> > origin/destination of these usually will be external catalog, usually in
> a
> > flattened(key/value) representation so I agree it is not as important as
> in
> > the aforementioned case. Nevertheless having a yaml based catalog or
> being
> > able to have e.g. yaml based snapshots of a catalog in my opinion is
> > appealing. At the same time cost of being able to have a nice
> > yaml/hocon/json representation is just adding a single suffix to a
> > single(at most 2 key + value) property. The question is between `format`
> =
> > `json` vs `format.kind` = `json`. That said I'd be slighty in favor of
> > doing it.
> >
> > Just to have a full picture. Both cases can be represented in yaml, but
> > the difference is significant:
> > format: 'json'
> > format.option: 'value'
> >
> > vs
> > format:
> > kind: 'json'
> >
> > option: 'value'
> >
> > Best,
> > Dawid
> >
> > On 29/04/2020 17:13, Flavio Pompermaier wrote:
> >
> > Personally I don't have any preference here.  Compliance wih standard
> YAML
> > parser is probably more important
> >
> > On Wed, Apr 29, 2020 at 5:10 PM Jark Wu  wrote:
> >
> >> From a user's perspective, I prefer the shorter one "format=json",
> because
> >> it's more concise and straightforward. The "kind" is redundant for
> users.
> >> Is there a real case requires to represent the configuration in JSON
> >> style?
> >> As far as I can see, I don't see such requirement, and everything works
> >> fine by now.
> >>
> >> So I'm in favor of "format=json". But if the community insist to follow
> >> code style on this, I'm also fine with the longer one.
> >>
> >> Btw, I also CC user mailing list to listen more user's feedback.
> Because I
> >> think this is relative to usability.
> >>
> >> Best,
> >> Jark
> >>
> >> On Wed, 29 Apr 2020 at 22:09, Chesnay Schepler 
> >> wrote:
> >>
> >> >  > Therefore, should we advocate instead:
> >> >  >
> >> >  > 'format.kind' = 'json',
> >> >  > 'format.fail-on-missing-field' = 'false'
> >> >
> >> > Yes. That's pretty much it.
> >> >
> >> > This is reasonable important to nail down as with such violations I
> >> > believe we could not actually switch to a standard YAML parser.
> >> >
> >> > On 29/04/2020 16:05, Timo Walther wrote:
> >> > > Hi everyone,
> >> > >
> >> > > discussions around ConfigOption seem to be very popular recently.
> So I
> >> > > would also like to get some opinions on a different topic.
> >> > >
> >> > > How do we represent hierarchies in ConfigOption? In FLIP-122, we
> >> > > agreed on the following DDL syntax:
> >> > >
> >> > > CREATE TABLE fs_table (
> >> > >  ...
> >> > > ) WITH (
> >> > >  'connector' = 'filesystem',
> >> > >  'path' = 'file:///path/to/whatever',
> >> > >  'format' = 'csv',
> >> > >  'format.allow-comments' = 'true',
> >> > >  'format.ignore-parse-errors' = 'true'
> >> > > );
> >> > >
> >> > > Of course this is slightly different from regular Flink core
> >> > > configuration but a connector still needs to be configured based on
> >> > > these options.
> >> > >
> >> > > However, I think this FLIP violates our code style guidelines
> because
> >> > >
> >> > > 'format' = 'json',
> >> > > 'format.fail-on-missing-field' = 'false'
> >> > >
> >> > > is an invalid hierarchy. `format` cannot be a string and a top-level
> >> > > object at the same time.
> >> > >
> >> > > We have similar problems in our runtime configuration:
> >> > >
> >> > > state.backend=
> >> > > state.backend.incremental

Re: [VOTE] FLIP-103: Better TM/JM Log Display

[Forward Xu]

+1

best,
Forward

Yadong Xie  于2020年4月28日周二 上午10:03写道：

> Hi all
>
> sorry for we have an issue that was not discovered in advance
>
> When users run multiple standalone TMs on the same host, the JM log list
> may confuse the users since all JM and TM logs are stored in the same
> folder, and they will appear on the same list.
>
> I have a discussion with Gary and Lining, a temporary workaround could be:
> show the main log and stdout file by default and make the log list a
> separate page.
>
> This workaround would allow us to roll out the feature while not changing
> existing behavior.
>
> What do you think?
>
> Till Rohrmann  于2020年2月29日周六 上午1:05写道：
>
> > Hi Yadong,
> >
> > Gary might be able to help with this effort.
> >
> > Cheers,
> > Till
> >
> > On Fri, Feb 28, 2020 at 11:08 AM Yadong Xie  wrote:
> >
> > > Hi Till
> > >
> > > In addition to the front end, the FLIP-103 would also include some rest
> > API
> > > changes.
> > > Could you assign one PMC or committer to help us review the pull
> request
> > > and drive this FLIP forward?
> > > We will start to create PR as soon as possible, thanks!
> > >
> > > Till Rohrmann  于2020年2月25日周二 下午6:03写道：
> > >
> > > > late +1 (binding)
> > > >
> > > > Cheers,
> > > > Till
> > > >
> > > > On Tue, Feb 25, 2020 at 4:46 AM Yadong Xie 
> > wrote:
> > > >
> > > > > Thanks all for the votes.
> > > > >
> > > > > So far, we have
> > > > >
> > > > > - 4 binding +1 votes (Kurt, Jark, jincheng, Zhu Zhu)
> > > > > - 5 non-binding +1 votes (Yang Wang, Xintong Song, lining, Yangze,
> > > > zhenya)
> > > > > - No -1 votes
> > > > >
> > > > > The voting time has past and there are enough +1 votes to consider
> > the
> > > > > FLIP-103 approved.
> > > > > Thank you all.
> > > > >
> > > > > Zhu Zhu  于2020年2月24日周一 下午3:56写道：
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > It can be very helpful to check GC state when GC logging is
> > enabled.
> > > > > >
> > > > > > Thanks,
> > > > > > Zhu Zhu
> > > > > >
> > > > > > zhenya Sun  于2020年2月24日周一 下午3:51写道：
> > > > > >
> > > > > > > 1
> > > > > > >
> > > > > > >
> > > > > > > able-sun
> > > > > > >
> > > > > > >
> > > > > > > On 02/24/2020 15:45，jincheng sun
> > wrote：
> > > > > > > +1(binding) Thank you for driving this Yadong!
> > > > > > >
> > > > > > > Best,
> > > > > > > Jincheng
> > > > > > >
> > > > > > > Yangze Guo  于2020年2月21日周五 下午3:08写道：
> > > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > It will make debugging more convenient and hence improve
> > usability.
> > > > > > >
> > > > > > > Best,
> > > > > > > Yangze Guo
> > > > > > >
> > > > > > > On Fri, Feb 21, 2020 at 2:50 PM Yang Wang <
> danrtsey...@gmail.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > Hi Jark,
> > > > > > >
> > > > > > > The GC log is not enabled by default. If the users want, he/she
> > > > should
> > > > > > > configure it by the following config option.
> > > > > > > env.java.opts.jobmanager: -XX:+PrintGCDetails
> > > -XX:+PrintGCDateStamps
> > > > > > > -Xloggc:/opt/flink/log/jobmanager-gc.log
> > -XX:+UseGCLogFileRotation
> > > > > > > -XX:NumberOfGCLogFiles=2 -XX:GCLogFileSize=512M
> > > > > > >
> > > > > > > Best,
> > > > > > > Yang
> > > > > > >
> > > > > > > Jark Wu  于2020年2月21日周五 下午2:19写道：
> > > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > A minor question: I didn't find the GC log in the POC web. Is
> it
> > > only
> > > > > > > generated when some configuration is enabled?
> > > > > > >
> > > > > > > Best,
> > > > > > > Jark
> > > > > > >
> > > > > > > On Fri, 21 Feb 2020 at 11:40, Kurt Young 
> > wrote:
> > > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Feb 21, 2020 at 11:25 AM lining jing <
> > > jinglini...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > +1 (non-binding)
> > > > > > > It lists all log files, the user could see the GC log.
> > > > > > >
> > > > > > > Xintong Song  于2020年2月21日周五 上午10:44写道：
> > > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > I like the ideas of having a list of all log files, and make
> them
> > > > > > > downloadable.
> > > > > > >
> > > > > > > Thank you~
> > > > > > >
> > > > > > > Xintong Song
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Feb 21, 2020 at 10:24 AM Yang Wang <
> > > > > > > danrtsey...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > +1 non-binding
> > > > > > >
> > > > > > > The log files list are shown in the new design. I like the
> > > > > > > idea.
> > > > > > > Since
> > > > > > > the
> > > > > > > user defined logs(e.g. my-log.file)
> > > > > > > could be found here now.
> > > > > > >
> > > > > > >
> > > > > > > Best,
> > > > > > > Yang
> > > > > > >
> > > > > > > Yadong Xie  于2020年2月20日周四 下午5:56写道：
> > > > > > >
> > > > > > > Hi all
> > > > > > >
> > > > > > > I want to start the vote for FLIP-103, which proposes to
> > > > > > > provide
> > > > > > > better
> > > > > > > user log display UI in task manager and job manager.
>

Re: New contributor

[Forward Xu]

welcome, Etienne

Best,
Forward

Hequn Cheng  于2020年4月25日周六 下午8:23写道：

> Welcome, Etienne :)
>
> Best,
> Hequn
>
> On Fri, Apr 24, 2020 at 10:04 PM Etienne Chauchot 
> wrote:
>
> > Hi Till,
> >
> > Looking forward too ...
> >
> > Thanks
> >
> > Etienne
> >
> > On 24/04/2020 15:09, Till Rohrmann wrote:
> > > Hi Etienne,
> > >
> > > welcome to the Flink community. Looking forward to working with you on
> > > Flink :-)
> > >
> > > Cheers,
> > > Till
> > >
> > > On Fri, Apr 24, 2020 at 11:20 AM Etienne Chauchot <
> echauc...@apache.org>
> > > wrote:
> > >
> > >> Hi everyone,
> > >>
> > >> Let me introduce myself, I'm Etienne Chauchot, I'm an Apache Beam
> > >> committer and a PMC member and I would like to start working on Flink
> as
> > >> well.
> > >>
> > >> For now I only did 3 simple Flink PRs but that will grow :)
> > >>
> > >> https://github.com/apache/flink/pull/11886
> > >>
> > >> https://github.com/apache/flink/pull/11740
> > >>
> > >> https://github.com/apache/flink/pull/11703
> > >>
> > >> Here is a link to my blog: https://echauchot.blogspot.com/
> > >>
> > >> Best.
> > >>
> > >> Etienne
> > >>
> > >>
> >
>

Re: [DISCUSS] JMX remote monitoring integration with Flink

[Forward Xu]

Hi  RongRong,
Thank you for bringing this discussion, it is indeed not appropriate to
occupy additional ports in the production environment to provide jmxrmi
services. I think [2] RestApi or JobManager/TaskManager UI is a good idea.

Best,
Forward

Rong Rong  于2020年3月13日周五 下午8:54写道：

> Hi All,
>
> Has anyone tried to manage production Flink applications through JMX remote
> monitoring & management[1]?
>
> We were experimenting to enable JMXRMI on Flink by default in production
> and would like to share some of our thoughts:
> ** Is there any straightforward way to dynamically allocate JMXRMI remote
> ports?*
>   - It is unrealistic to use JMXRMI static port in production environment,
> however we have to go all around the logging system to make the dynamic
> remote port number printed out in the log files - this seems very
> inconvenient.
>   - I think it would be very handy if we can show the JMXRMI remote
> information on JobManager/TaskManager UI, or via REST API. (I am thinking
> about something similar to [2])
>
> ** Is there any performance overhead enabling JMX for a Flink application?*
>   - We haven't seen any significant performance impact in our experiments.
> However the experiment is not that well-rounded and the observation is
> inconclusive.
>   - I was wondering would it be a good idea to put some benchmark in the
> regression tests[3] to see what's the overhead would be?
>
> It would be highly appreciated if anyone could share some experiences or
> provide any suggestions in how we can improve the JMX remote integration
> with Flink.
>
>
> Thanks,
> Rong
>
>
> [1]
>
> https://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html
> [2]
> https://samza.apache.org/learn/documentation/0.14/jobs/web-ui-rest-api.html
> [3] http://codespeed.dak8s.net:8000/
>

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

[Forward Xu]

Congratulations! Jingsong


Best,

Forward

Terry Wang  于2020年3月2日周一 上午11:10写道：

> Congratulations,  well deserved.!
>
> Best,
> Terry Wang
>
>
>
> > 2020年2月27日 10:50，Yuan Mei  写道：
> >
> > Congrats!
> >
> > Best
> > Yuan
> >
> > On Thu, Feb 27, 2020 at 8:48 AM Guowei Ma  wrote:
> >
> >> Congratulations !!
> >> Best,
> >> Guowei
> >>
> >>
> >> Yun Tang  于2020年2月27日周四 上午2:11写道：
> >>
> >>> Congratulations and well deserved!
> >>>
> >>>
> >>> Best
> >>> Yun Tang
> >>> 
> >>> From: Canbin Zheng 
> >>> Sent: Monday, February 24, 2020 16:07
> >>> To: dev 
> >>> Subject: Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer
> >>>
> >>> Congratulations !!
> >>>
> >>> Dawid Wysakowicz  于2020年2月24日周一 下午3:55写道：
> >>>
>  Congratulations Jingsong!
> 
>  Best,
> 
>  Dawid
> 
>  On 24/02/2020 08:12, zhenya Sun wrote:
> > Congratulations！！！
> > | |
> > zhenya Sun
> > |
> > |
> > toke...@126.com
> > |
> > 签名由网易邮箱大师定制
> >
> >
> > On 02/24/2020 14:35，Yu Li wrote：
> > Congratulations Jingsong! Well deserved.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Mon, 24 Feb 2020 at 14:10, Congxian Qiu 
>  wrote:
> >
> > Congratulations Jingsong!
> >
> > Best,
> > Congxian
> >
> >
> > jincheng sun  于2020年2月24日周一 下午1:38写道：
> >
> > Congratulations Jingsong!
> >
> > Best,
> > Jincheng
> >
> >
> > Zhu Zhu  于2020年2月24日周一 上午11:55写道：
> >
> > Congratulations Jingsong!
> >
> > Thanks,
> > Zhu Zhu
> >
> > Fabian Hueske  于2020年2月22日周六 上午1:30写道：
> >
> > Congrats Jingsong!
> >
> > Cheers, Fabian
> >
> > Am Fr., 21. Feb. 2020 um 17:49 Uhr schrieb Rong Rong <
> > walter...@gmail.com>:
> >
> > Congratulations Jingsong!!
> >
> > Cheers,
> > Rong
> >
> > On Fri, Feb 21, 2020 at 8:45 AM Bowen Li 
> >> wrote:
> >
> > Congrats, Jingsong!
> >
> > On Fri, Feb 21, 2020 at 7:28 AM Till Rohrmann  >
> > wrote:
> >
> > Congratulations Jingsong!
> >
> > Cheers,
> > Till
> >
> > On Fri, Feb 21, 2020 at 4:03 PM Yun Gao 
> > wrote:
> >
> > Congratulations Jingsong!
> >
> > Best,
> > Yun
> >
> > --
> > From:Jingsong Li 
> > Send Time:2020 Feb. 21 (Fri.) 21:42
> > To:Hequn Cheng 
> > Cc:Yang Wang ; Zhijiang <
> > wangzhijiang...@aliyun.com>; Zhenghua Gao ;
> > godfrey
> > he ; dev ; user <
> > u...@flink.apache.org>
> > Subject:Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer
> >
> > Thanks everyone~
> >
> > It's my pleasure to be part of the community. I hope I can make a
> > better
> > contribution in future.
> >
> > Best,
> > Jingsong Lee
> >
> > On Fri, Feb 21, 2020 at 2:48 PM Hequn Cheng 
> > wrote:
> > Congratulations Jingsong! Well deserved.
> >
> > Best,
> > Hequn
> >
> > On Fri, Feb 21, 2020 at 2:42 PM Yang Wang 
> > wrote:
> > Congratulations！Jingsong. Well deserved.
> >
> >
> > Best,
> > Yang
> >
> > Zhijiang  于2020年2月21日周五 下午1:18写道：
> > Congrats Jingsong! Welcome on board!
> >
> > Best,
> > Zhijiang
> >
> > --
> > From:Zhenghua Gao 
> > Send Time:2020 Feb. 21 (Fri.) 12:49
> > To:godfrey he 
> > Cc:dev ; user 
> > Subject:Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer
> >
> > Congrats Jingsong！
> >
> >
> > *Best Regards,*
> > *Zhenghua Gao*
> >
> >
> > On Fri, Feb 21, 2020 at 11:59 AM godfrey he 
> > wrote:
> > Congrats Jingsong! Well deserved.
> >
> > Best,
> > godfrey
> >
> > Jeff Zhang  于2020年2月21日周五 上午11:49写道：
> > Congratulations！Jingsong. You deserve it
> >
> > wenlong.lwl  于2020年2月21日周五 上午11:43写道：
> > Congrats Jingsong!
> >
> > On Fri, 21 Feb 2020 at 11:41, Dian Fu 
> > wrote:
> >
> > Congrats Jingsong!
> >
> > 在 2020年2月21日，上午11:39，Jark Wu  写道：
> >
> > Congratulations Jingsong! Well deserved.
> >
> > Best,
> > Jark
> >
> > On Fri, 21 Feb 2020 at 11:32, zoudan  wrote:
> >
> > Congratulations! Jingsong
> >
> >
> > Best,
> > Dan Zou
> >
> >
> >
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
> >
> >
> > --
> > Best, Jingsong Lee
> >
> >
> >
> >
> >
> >
> 
> 
> >>>
> >>
>
>

[VOTE][RESULT] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

[Forward Xu]

 Hi all,

 Votes on FLIP-90 have been supported by 3 commitors. Thanks everyone for
voting. Related discussions in [1] & [2] & [3]. The results of this vote
are in [4] & [5].

 Best,
 Forward

[1]
https://docs.google.com/document/d/1JfaFYIFOAY8P2pFhOYNCQ9RTzwF4l85_bnTvImOLKMk/edit#heading=h.76mb88ca6yjp
[2]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=141724550
[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-JSON-functions-in-Flink-SQL-td32674.html
[4]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-90-Support-SQL-2016-2017-JSON-functions-in-Flink-SQL-tt36532.html
[5]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-90-Support-SQL-2016-2017-JSON-functions-in-Flink-SQL-tt36341.html

Re: [VOTE] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

[Forward Xu]

Hi Danny,
Thank you very much.

Best,
Forward

Danny Chan  于2020年1月10日周五 上午11:08写道：

> Thanks Forward ~
> +1 from my side and would review your Calcite PR this weekend :) Overall
> it looks good, and I believe we can merge it soon ~
>
> Best,
> Danny Chan
> 在 2020年1月10日 +0800 AM11:04，Jark Wu ，写道：
> > Thanks Forward for driving this,
> >
> > The design doc looks very good to me.
> > +1 from my side.
> >
> > Best,
> > Jark
> >
> > On Thu, 9 Jan 2020 at 20:12, Forward Xu  wrote:
> >
> > > Hi all,
> > >
> > > Listened to the opinion of Timo since the last discussion and updated
> the
> > > document [1] Optimized the passing parameters of JSON table API. Added
> > > return type when describing each JSON function. It makes the
> documentation
> > > more clear. So I again vote of FLIP-90 [2] since that we have reached
> an
> > > agreement on the design in the discussion thread [3].
> > > This vote will be open for at least 72 hours. Unless there is an
> objection,
> > > I will try to close it by January 12, 2020 08:00 UTC if we have
> received
> > > sufficient votes.
> > > Best,
> > > ForwardXu
> > >
> > > [1]
> > >
> > >
> https://docs.google.com/document/d/1JfaFYIFOAY8P2pFhOYNCQ9RTzwF4l85_bnTvImOLKMk/edit#heading=h.76mb88ca6yjp
> > > [2]
> > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=141724550
> > > [3]
> > >
> > >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-JSON-functions-in-Flink-SQL-td32674.html
> > >
>

[VOTE] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

[Forward Xu]

Hi all,

Listened to the opinion of Timo since the last discussion and updated the
document [1] Optimized the passing parameters of JSON table API. Added
return type when describing each JSON function. It makes the documentation
more clear. So I again vote of FLIP-90 [2] since that we have reached an
agreement on the design in the discussion thread [3].
This vote will be open for at least 72 hours. Unless there is an objection,
I will try to close it by January 12, 2020 08:00 UTC if we have received
sufficient votes.
Best,
ForwardXu

[1]
https://docs.google.com/document/d/1JfaFYIFOAY8P2pFhOYNCQ9RTzwF4l85_bnTvImOLKMk/edit#heading=h.76mb88ca6yjp
[2]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=141724550
[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-JSON-functions-in-Flink-SQL-td32674.html

[VOTE] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

[Forward Xu]

Hi all,

Listened to the opinion of timo since the last discussion and updated the
document [1] Optimized the passing parameters of json table api. Added
return type when describing each json function. Makes the documentation
more clear. So i again vote of FLIP-90 [2] since that we have reached an
agreement on the design in the discussion thread [3].
This vote will be open for at least 72 hours. Unless there is an
objection,I will try to close it by January 13, 2020 08:00 UTC if we have
received sufficient votes.
Best,
ForwardXu

[1]
https://docs.google.com/document/d/1JfaFYIFOAY8P2pFhOYNCQ9RTzwF4l85_bnTvImOLKMk/edit#heading=h.76mb88ca6yjp
[2]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=141724550
[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-JSON-functions-in-Flink-SQL-td32674.html

Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

[Forward Xu]

hi，Dann，Thanks you very much.

Best,
Forward

Danny Chan  于2020年1月8日周三 下午2:25写道：

> Thanks Forward ~
>
> I would help to view code from the Calcite side.
>
> Best,
> Danny Chan
> 在 2020年1月7日 +0800 PM2:01，Forward Xu ，写道：
> > Thanks Jark for checking the doc. hi,Timo, please help to check to see if
> > there is anything else to add.
> >
> > Best,
> > Forward
> >
> > Jark Wu  于2020年1月6日周一 下午2:58写道：
> >
> > > Thanks Forward for the updating. It is much more clearer now about the
> > > returning type, especially JSON_VALUE.
> > > The design doc looks good to me now.
> > >
> > > Best,
> > > Jark
> > >
> > > On Fri, 3 Jan 2020 at 21:42, Forward Xu 
> wrote:
> > >
> > > > Hi Timo, Jack,
> > > > Well, I added additional column to describe the return type of each
> > > > function and
> > > > updated the google doc.
> > > >
> > > > Best,
> > > > Forward
> > > >
> > > > Jark Wu  于2020年1月3日周五 下午5:48写道：
> > > >
> > > > > Hi Timo,
> > > > >
> > > > > That's a good point.
> > > > > We didn't introduce any new types. We will use the function
> definition
> > > > > defined by Calcite[1].
> > > > > So all the functions return STRING/BOOLEAN.
> > > > >
> > > > > Hi Forward,
> > > > > I think we may need an additional column to describe the return
> type of
> > > > > each function.
> > > > >
> > > > > Best,
> > > > > Jark
> > > > >
> > > > > [1]:
> > > > >
> > > > >
> > > >
> > >
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlJsonQueryFunction.java
> > > > >
> > > > > On Fri, 3 Jan 2020 at 17:30, Timo Walther 
> wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > sorry for jumping into the discussion so late. I had a quick
> look at
> > > > the
> > > > > > FLIP. It looks very nice and detailed. I have one question that I
> > > could
> > > > > > not find in the FLIP itself. Maybe it is hidden in the long
> > > discussion
> > > > > > thread.
> > > > > >
> > > > > > What are the return types of all functions? Do we introduce new
> types
> > > > > > with this FLIP? Also the RAW types should be avoided. Do all
> > > functions
> > > > > > return STRING/BOOLEAN?
> > > > > >
> > > > > > Thanks,
> > > > > > Timo
> > > > > >
> > > > > >
> > > > > > On 31.12.19 09:39, Hequn Cheng wrote:
> > > > > > > Thanks a lot for the update. +1 to start a vote.
> > > > > > >
> > > > > > > On Tue, Dec 31, 2019 at 2:38 PM Forward Xu <
> forwardxu...@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Jark, Hequn,
> > > > > > > >
> > > > > > > > I have updated the documentation.
> > > > > > > >
> > > > > > > > Best,
> > > > > > > >
> > > > > > > > Forward
> > > > > > > >
> > > > > > > > Forward Xu  于2019年12月29日周日 下午4:01写道：
> > > > > > > >
> > > > > > > > > Hi Jark, Hequn,
> > > > > > > > >
> > > > > > > > > Thank you very much, Introducing new `TableSymbol`s sounds
> like a
> > > > > good
> > > > > > > > > idea. +1 for the proposal.
> > > > > > > > >
> > > > > > > > > I think this idea is good, I will add this in the
> documentation.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Best, Forward
> > > > > > > > >
> > > > > > > > > Hequn Cheng  于2019年12月29日周日
> 下午3:41写道：
> > > > > > > > >
> > > > > > > > > > Hi Jark,
> > > > > > > > > >
> > > > > > > > > > In

Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

[Forward Xu]

Thanks Jark for checking the doc. hi,Timo, please help to check to see if
there is anything else to add.

Best,
Forward

Jark Wu  于2020年1月6日周一 下午2:58写道：

> Thanks Forward for the updating. It is much more clearer now about the
> returning type, especially JSON_VALUE.
> The design doc looks good to me now.
>
> Best,
> Jark
>
> On Fri, 3 Jan 2020 at 21:42, Forward Xu  wrote:
>
> > Hi Timo, Jack,
> > Well, I added additional column to describe the return type of each
> > function and
> > updated the google doc.
> >
> > Best,
> > Forward
> >
> > Jark Wu  于2020年1月3日周五 下午5:48写道：
> >
> > > Hi Timo,
> > >
> > > That's a good point.
> > > We didn't introduce any new types. We will use the function definition
> > > defined by Calcite[1].
> > > So all the functions return STRING/BOOLEAN.
> > >
> > > Hi Forward,
> > > I think we may need an additional column to describe the return type of
> > > each function.
> > >
> > > Best,
> > > Jark
> > >
> > > [1]:
> > >
> > >
> >
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlJsonQueryFunction.java
> > >
> > > On Fri, 3 Jan 2020 at 17:30, Timo Walther  wrote:
> > >
> > > > Hi,
> > > >
> > > > sorry for jumping into the discussion so late. I had a quick look at
> > the
> > > > FLIP. It looks very nice and detailed. I have one question that I
> could
> > > > not find in the FLIP itself. Maybe it is hidden in the long
> discussion
> > > > thread.
> > > >
> > > > What are the return types of all functions? Do we introduce new types
> > > > with this FLIP? Also the RAW types should be avoided. Do all
> functions
> > > > return STRING/BOOLEAN?
> > > >
> > > > Thanks,
> > > > Timo
> > > >
> > > >
> > > > On 31.12.19 09:39, Hequn Cheng wrote:
> > > > > Thanks a lot for the update. +1 to start a vote.
> > > > >
> > > > > On Tue, Dec 31, 2019 at 2:38 PM Forward Xu  >
> > > > wrote:
> > > > >
> > > > >> Hi Jark, Hequn,
> > > > >>
> > > > >> I have updated the documentation.
> > > > >>
> > > > >> Best,
> > > > >>
> > > > >> Forward
> > > > >>
> > > > >> Forward Xu  于2019年12月29日周日 下午4:01写道：
> > > > >>
> > > > >>> Hi Jark, Hequn,
> > > > >>>
> > > > >>> Thank you very much, Introducing new `TableSymbol`s sounds like a
> > > good
> > > > >>> idea. +1 for the proposal.
> > > > >>>
> > > > >>> I think this idea is good, I will add this in the documentation.
> > > > >>>
> > > > >>>
> > > > >>> Best, Forward
> > > > >>>
> > > > >>> Hequn Cheng  于2019年12月29日周日 下午3:41写道：
> > > > >>>
> > > > >>>> Hi Jark,
> > > > >>>>
> > > > >>>> Introducing new `TableSymbol`s sounds like a good idea. +1 for
> the
> > > > >>>> proposal.
> > > > >>>> @ForwardXu what do you think? Would be great if the document can
> > be
> > > > >>>> updated
> > > > >>>> accordingly.
> > > > >>>>
> > > > >>>> Best, Hequn
> > > > >>>>
> > > > >>>>
> > > > >>>> On Fri, Dec 27, 2019 at 4:14 PM Jark Wu 
> wrote:
> > > > >>>>
> > > > >>>>> Thanks for looking into the design Hequn. I agree it would be
> > great
> > > > to
> > > > >>>>> have a full story design.
> > > > >>>>>
> > > > >>>>> For the ON ERROR and ON EMPTY clause in Table API, some initial
> > > > >>>>> thoughts in my mind is that
> > > > >>>>> we can introduce some new `TableSymbol`s as the second
> parameter
> > of
> > > > >> json
> > > > >>>>> function, e.g. `JsonErrorStrategy`.
> > > > >>>>>
> > > > >>>>> For example,
> > >

Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

[Forward Xu]

Hi Timo, Jack,
Well, I added additional column to describe the return type of each
function and
updated the google doc.

Best,
Forward

Jark Wu  于2020年1月3日周五 下午5:48写道：

> Hi Timo,
>
> That's a good point.
> We didn't introduce any new types. We will use the function definition
> defined by Calcite[1].
> So all the functions return STRING/BOOLEAN.
>
> Hi Forward,
> I think we may need an additional column to describe the return type of
> each function.
>
> Best,
> Jark
>
> [1]:
>
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlJsonQueryFunction.java
>
> On Fri, 3 Jan 2020 at 17:30, Timo Walther  wrote:
>
> > Hi,
> >
> > sorry for jumping into the discussion so late. I had a quick look at the
> > FLIP. It looks very nice and detailed. I have one question that I could
> > not find in the FLIP itself. Maybe it is hidden in the long discussion
> > thread.
> >
> > What are the return types of all functions? Do we introduce new types
> > with this FLIP? Also the RAW types should be avoided. Do all functions
> > return STRING/BOOLEAN?
> >
> > Thanks,
> > Timo
> >
> >
> > On 31.12.19 09:39, Hequn Cheng wrote:
> > > Thanks a lot for the update. +1 to start a vote.
> > >
> > > On Tue, Dec 31, 2019 at 2:38 PM Forward Xu 
> > wrote:
> > >
> > >> Hi Jark, Hequn,
> > >>
> > >> I have updated the documentation.
> > >>
> > >> Best,
> > >>
> > >> Forward
> > >>
> > >> Forward Xu  于2019年12月29日周日 下午4:01写道：
> > >>
> > >>> Hi Jark, Hequn,
> > >>>
> > >>> Thank you very much, Introducing new `TableSymbol`s sounds like a
> good
> > >>> idea. +1 for the proposal.
> > >>>
> > >>> I think this idea is good, I will add this in the documentation.
> > >>>
> > >>>
> > >>> Best, Forward
> > >>>
> > >>> Hequn Cheng  于2019年12月29日周日 下午3:41写道：
> > >>>
> > >>>> Hi Jark,
> > >>>>
> > >>>> Introducing new `TableSymbol`s sounds like a good idea. +1 for the
> > >>>> proposal.
> > >>>> @ForwardXu what do you think? Would be great if the document can be
> > >>>> updated
> > >>>> accordingly.
> > >>>>
> > >>>> Best, Hequn
> > >>>>
> > >>>>
> > >>>> On Fri, Dec 27, 2019 at 4:14 PM Jark Wu  wrote:
> > >>>>
> > >>>>> Thanks for looking into the design Hequn. I agree it would be great
> > to
> > >>>>> have a full story design.
> > >>>>>
> > >>>>> For the ON ERROR and ON EMPTY clause in Table API, some initial
> > >>>>> thoughts in my mind is that
> > >>>>> we can introduce some new `TableSymbol`s as the second parameter of
> > >> json
> > >>>>> function, e.g. `JsonErrorStrategy`.
> > >>>>>
> > >>>>> For example,
> > >>>>>
> > >>>>> JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
> > >>>>> == is equal to Table API ==>
> > >>>>> v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
> > >>>>>
> > >>>>> Best,
> > >>>>> Jark
> > >>>>>
> > >>>>>
> > >>>>> On Thu, 26 Dec 2019 at 23:37, Hequn Cheng 
> > >> wrote:
> > >>>>>
> > >>>>>> Hi Jark & ForwardXu,
> > >>>>>>
> > >>>>>> The design doc looks very nice! Only some minor feedback from my
> > >> side.
> > >>>>>>
> > >>>>>> As calcite has already implemented the JSON functions, I would
> > >> suppose
> > >>>>>> the semantics and implementation are right for SQL.
> > >>>>>>
> > >>>>>> For TableAPI, I think the most important is to keep align with the
> > >>>>>> SQL(which has also been mentioned by Jark in the previous
> > >> discussion).
> > >>>> Have
> > >>>>>> an equivalent feature set for all APIs and maintain it otherwise
> > >>>> confusion
> > >>>>>> increases especially when more and more functions are added. The
> > >>>> document
> > >>>>>> has documented how to support TableAPI. I think this is very good!
> > >> And
> > >>>> it
> > >>>>>> would be better to also include ON ERROR or ON EMPTY for Table
> API.
> > >> We
> > >>>> can
> > >>>>>> implement these features step by step, but maybe we should design
> > all
> > >>>> these
> > >>>>>> once for all to avoid API changes later. Meanwhile, these features
> > >> are
> > >>>> also
> > >>>>>> commonly required by users.
> > >>>>>>
> > >>>>>> Would be great to also have your opinions!
> > >>>>>>
> > >>>>>> Best,
> > >>>>>> Hequn
> > >>>>>>
> > >>>>>>
> > >>>>>> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu 
> wrote:
> > >>>>>>
> > >>>>>>> Hi Forward,
> > >>>>>>>
> > >>>>>>> Thanks for creating the FLIP. +1 to start a vote.
> > >>>>>>>
> > >>>>>>>   @Hequn Cheng  @Kurt Young <
> > ykt...@gmail.com>
> > >> ,
> > >>>>>>> could you help to review the design doc too?
> > >>>>>>>
> > >>>>>>> Best,
> > >>>>>>> Jark
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Mon, 23 Dec 2019 at 10:10, tison 
> wrote:
> > >>>>>>>
> > >>>>>>>> modified:
> > >>>>>>>>
> > >>>>>>>>
> > >>>>
> > >>
> >
> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
> > >>>>>>>>
> > >>>>>>>
> > >>>>
> > >>>
> > >>
> > >
> >
> >
>

[VOTE] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

[Forward Xu]

Hi all,

I'd like to start the vote of FLIP-90 [1] since that we have reached an
agreement on the design in the discussion thread [2].

This vote will be open for at least 72 hours. Unless there is an objection,
I will try to close it by January 3, 2020 08:00 UTC if we have received
sufficient votes.

Best,
ForwardXu

[1]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=141724550
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-JSON-functions-in-Flink-SQL-td32674.html

Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

[Forward Xu]

Hi Jark, Hequn,

I have updated the documentation.

Best,

Forward

Forward Xu  于2019年12月29日周日 下午4:01写道：

> Hi Jark, Hequn,
>
> Thank you very much, Introducing new `TableSymbol`s sounds like a good
> idea. +1 for the proposal.
>
> I think this idea is good, I will add this in the documentation.
>
>
> Best, Forward
>
> Hequn Cheng  于2019年12月29日周日 下午3:41写道：
>
>> Hi Jark,
>>
>> Introducing new `TableSymbol`s sounds like a good idea. +1 for the
>> proposal.
>> @ForwardXu what do you think? Would be great if the document can be
>> updated
>> accordingly.
>>
>> Best, Hequn
>>
>>
>> On Fri, Dec 27, 2019 at 4:14 PM Jark Wu  wrote:
>>
>> > Thanks for looking into the design Hequn. I agree it would be great to
>> > have a full story design.
>> >
>> > For the ON ERROR and ON EMPTY clause in Table API, some initial
>> > thoughts in my mind is that
>> > we can introduce some new `TableSymbol`s as the second parameter of json
>> > function, e.g. `JsonErrorStrategy`.
>> >
>> > For example,
>> >
>> > JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
>> > == is equal to Table API ==>
>> > v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
>> >
>> > Best,
>> > Jark
>> >
>> >
>> > On Thu, 26 Dec 2019 at 23:37, Hequn Cheng  wrote:
>> >
>> >> Hi Jark & ForwardXu,
>> >>
>> >> The design doc looks very nice! Only some minor feedback from my side.
>> >>
>> >> As calcite has already implemented the JSON functions, I would suppose
>> >> the semantics and implementation are right for SQL.
>> >>
>> >> For TableAPI, I think the most important is to keep align with the
>> >> SQL(which has also been mentioned by Jark in the previous discussion).
>> Have
>> >> an equivalent feature set for all APIs and maintain it otherwise
>> confusion
>> >> increases especially when more and more functions are added. The
>> document
>> >> has documented how to support TableAPI. I think this is very good! And
>> it
>> >> would be better to also include ON ERROR or ON EMPTY for Table API. We
>> can
>> >> implement these features step by step, but maybe we should design all
>> these
>> >> once for all to avoid API changes later. Meanwhile, these features are
>> also
>> >> commonly required by users.
>> >>
>> >> Would be great to also have your opinions!
>> >>
>> >> Best,
>> >> Hequn
>> >>
>> >>
>> >> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu  wrote:
>> >>
>> >>> Hi Forward,
>> >>>
>> >>> Thanks for creating the FLIP. +1 to start a vote.
>> >>>
>> >>>  @Hequn Cheng  @Kurt Young  ,
>> >>> could you help to review the design doc too?
>> >>>
>> >>> Best,
>> >>> Jark
>> >>>
>> >>>
>> >>> On Mon, 23 Dec 2019 at 10:10, tison  wrote:
>> >>>
>> >>>> modified:
>> >>>>
>> >>>>
>> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
>> >>>>
>> >>>
>>
>

Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

[Forward Xu]

Hi Jark, Hequn,

Thank you very much, Introducing new `TableSymbol`s sounds like a good
idea. +1 for the proposal.

I think this idea is good, I will add this in the documentation.


Best, Forward

Hequn Cheng  于2019年12月29日周日 下午3:41写道：

> Hi Jark,
>
> Introducing new `TableSymbol`s sounds like a good idea. +1 for the
> proposal.
> @ForwardXu what do you think? Would be great if the document can be updated
> accordingly.
>
> Best, Hequn
>
>
> On Fri, Dec 27, 2019 at 4:14 PM Jark Wu  wrote:
>
> > Thanks for looking into the design Hequn. I agree it would be great to
> > have a full story design.
> >
> > For the ON ERROR and ON EMPTY clause in Table API, some initial
> > thoughts in my mind is that
> > we can introduce some new `TableSymbol`s as the second parameter of json
> > function, e.g. `JsonErrorStrategy`.
> >
> > For example,
> >
> > JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
> > == is equal to Table API ==>
> > v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
> >
> > Best,
> > Jark
> >
> >
> > On Thu, 26 Dec 2019 at 23:37, Hequn Cheng  wrote:
> >
> >> Hi Jark & ForwardXu,
> >>
> >> The design doc looks very nice! Only some minor feedback from my side.
> >>
> >> As calcite has already implemented the JSON functions, I would suppose
> >> the semantics and implementation are right for SQL.
> >>
> >> For TableAPI, I think the most important is to keep align with the
> >> SQL(which has also been mentioned by Jark in the previous discussion).
> Have
> >> an equivalent feature set for all APIs and maintain it otherwise
> confusion
> >> increases especially when more and more functions are added. The
> document
> >> has documented how to support TableAPI. I think this is very good! And
> it
> >> would be better to also include ON ERROR or ON EMPTY for Table API. We
> can
> >> implement these features step by step, but maybe we should design all
> these
> >> once for all to avoid API changes later. Meanwhile, these features are
> also
> >> commonly required by users.
> >>
> >> Would be great to also have your opinions!
> >>
> >> Best,
> >> Hequn
> >>
> >>
> >> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu  wrote:
> >>
> >>> Hi Forward,
> >>>
> >>> Thanks for creating the FLIP. +1 to start a vote.
> >>>
> >>>  @Hequn Cheng  @Kurt Young  ,
> >>> could you help to review the design doc too?
> >>>
> >>> Best,
> >>> Jark
> >>>
> >>>
> >>> On Mon, 23 Dec 2019 at 10:10, tison  wrote:
> >>>
>  modified:
> 
> 
> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
> 
> >>>
>

[jira] [Created] (FLINK-15356) Add applicationId to existing flink metrics running on yarn

[Forward Xu (Jira)]

Forward Xu created FLINK-15356:
--

 Summary: Add applicationId to existing flink metrics running on 
yarn
 Key: FLINK-15356
 URL: https://issues.apache.org/jira/browse/FLINK-15356
 Project: Flink
  Issue Type: Improvement
  Components: Runtime / Metrics
Reporter: Forward Xu


When sending metrics to Prometheus, these systems have only the Flink job ID, 
and the Flink job ID is UUID, which cannot be associated with the application 
job on the yarn. Therefore, we need to increase the applicationId when running 
on yarn. This helps us to accurately find the corresponding job on yarn when 
the metric is abnormal.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Re: [DISCUSS] Support JSON functions in Flink SQL

[Forward Xu]

Hi Jark，
  Thanks for your help. I convert it into FLIP wiki already.
Best,
Forward

Jark Wu  于2019年12月20日周五 下午11:09写道：

> Hi Forward,
>
> Thanks for updating the documentation. I think the design doc looks good to
> me now. I think you can convert it into FLIP wiki and start a vote then.
>
> Best,Jark
>
> Jark Wu 于2019年12月20日 周五23:07写道：
>
> > Hi Forward,
> >
> > Thanks for updating the documentation. I think it is
> >
> > On Mon, 2 Dec 2019 at 14:21, Forward Xu  wrote:
> >
> >> Hi Jark,
> >> Thank you very much，I will improve this document as soon as possible.
> >> My confluence username is ForwardXu.
> >>
> >> Best,
> >> Forward
> >>
> >> Jark Wu  于2019年12月2日周一 上午11:56写道：
> >>
> >> > Hi Forward,
> >> >
> >> > Sorry for the late reply.
> >> > As I said before, it would be better to include the JSON functions API
> >> for
> >> > Table API.
> >> > For example, what the "json_exist", "json_value", "json_query", etc...
> >> > functions in Table API looks like.
> >> > And it would be better to follow FLIP template [1] which include
> "Public
> >> > Interface" (brief list of public interface), and "Proposed changes"
> >> > (detailed proposal).
> >> >
> >> > Once the design doc looks good, you can update it to FLIP and start a
> >> vote.
> >> >
> >> > Btw, what's your username in confluence?
> >> >
> >> >
> >> > Best,
> >> > Jark
> >> >
> >> >
> >> > [1]: https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
> >> >
> >> >
> >> > On Mon, 2 Dec 2019 at 11:44, Forward Xu 
> wrote:
> >> >
> >> > > Hi Jingsong Lee :
> >> > >
> >> > > Thank you very much, I need to apply for FLIP permission. Do I need
> to
> >> > > create a FLIP for this?
> >> > >
> >> > > Best,
> >> > > Forward
> >> > >
> >> > > Jingsong Li  于2019年12月2日周一 上午11:40写道：
> >> > >
> >> > > > Hi Forward:
> >> > > >
> >> > > > Document looks good to me.
> >> > > > I think you can just start doing this.
> >> > > > They all work very independently, so I don't think there's any
> >> obvious
> >> > > > blocking.
> >> > > >
> >> > > > Best,
> >> > > > Jingsong Lee
> >> > > >
> >> > > > On Sat, Nov 30, 2019 at 10:59 AM Forward Xu <
> forwardxu...@gmail.com
> >> >
> >> > > > wrote:
> >> > > >
> >> > > > > Hi everyone, It's been a long time since I started this
> >> discussion.
> >> > Do
> >> > > > you
> >> > > > > have anything to add and improve?
> >> > > > > Best,
> >> > > > > Forward
> >> > > > >
> >> > > > > Forward Xu  于2019年9月22日周日 下午6:30写道：
> >> > > > >
> >> > > > > > Hi Jack,
> >> > > > > > Thank you very much for your reply, google doc I have updated,
> >> and
> >> > > some
> >> > > > > of
> >> > > > > > your questions I replied.
> >> > > > > >  In addition, I want to apply for Flip permissions for this
> >> > purpose.
> >> > > > > >
> >> > > > > > Best,
> >> > > > > > Forward
> >> > > > > >
> >> > > > > > Jark Wu  于2019年9月20日周五 下午9:53写道：
> >> > > > > >
> >> > > > > >> Hi Forward,
> >> > > > > >>
> >> > > > > >> Sorry for the late reply. I have went through the design doc
> >> and I
> >> > > > think
> >> > > > > >> it
> >> > > > > >> is very nice.
> >> > > > > >>
> >> > > > > >> Here are my thoughts and suggestions:
> >> > > > > >>
> >> > > > > >> 0) I think support JSON functions in SQL is not complicated.
> >> > Because
> >> > > > > >

[DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

[Forward Xu]

Hi everybody,


I'd like to kick off a discussion on FLIP-90: Support SQL 2016-2017 JSON
functions in Flink SQL.


Implement Support SQL 2016-2017 JSON functions in Flink SQL[1].



Would love to hear your thoughts.


[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-90%3A+Support+SQL+2016-2017+JSON+functions+in+Flink+SQL


Best,

ForwardXu

>

[DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

[Forward Xu]

Hi everybody,


I'd like to kick off a discussion on FLIP-90: Support SQL 2016-2017 JSON
functions in Flink SQL.


Implement Support SQL 2016-2017 JSON functions in Flink SQL[1].



Would love to hear your thoughts.


[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-90%3A+Support+SQL+2016-2017+JSON+functions+in+Flink+SQL



Best,

ForwardXu

>

Re: Re: [DISCUSS] Support JSON functions in Flink SQL

[Forward Xu]

Hi Jark,
Thank you very much，I will improve this document as soon as possible.
My confluence username is ForwardXu.

Best,
Forward

Jark Wu  于2019年12月2日周一 上午11:56写道：

> Hi Forward,
>
> Sorry for the late reply.
> As I said before, it would be better to include the JSON functions API for
> Table API.
> For example, what the "json_exist", "json_value", "json_query", etc...
> functions in Table API looks like.
> And it would be better to follow FLIP template [1] which include "Public
> Interface" (brief list of public interface), and "Proposed changes"
> (detailed proposal).
>
> Once the design doc looks good, you can update it to FLIP and start a vote.
>
> Btw, what's your username in confluence?
>
>
> Best,
> Jark
>
>
> [1]: https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
>
>
> On Mon, 2 Dec 2019 at 11:44, Forward Xu  wrote:
>
> > Hi Jingsong Lee :
> >
> > Thank you very much, I need to apply for FLIP permission. Do I need to
> > create a FLIP for this?
> >
> > Best,
> > Forward
> >
> > Jingsong Li  于2019年12月2日周一 上午11:40写道：
> >
> > > Hi Forward:
> > >
> > > Document looks good to me.
> > > I think you can just start doing this.
> > > They all work very independently, so I don't think there's any obvious
> > > blocking.
> > >
> > > Best,
> > > Jingsong Lee
> > >
> > > On Sat, Nov 30, 2019 at 10:59 AM Forward Xu 
> > > wrote:
> > >
> > > > Hi everyone, It's been a long time since I started this discussion.
> Do
> > > you
> > > > have anything to add and improve?
> > > > Best,
> > > > Forward
> > > >
> > > > Forward Xu  于2019年9月22日周日 下午6:30写道：
> > > >
> > > > > Hi Jack,
> > > > > Thank you very much for your reply, google doc I have updated, and
> > some
> > > > of
> > > > > your questions I replied.
> > > > >  In addition, I want to apply for Flip permissions for this
> purpose.
> > > > >
> > > > > Best,
> > > > > Forward
> > > > >
> > > > > Jark Wu  于2019年9月20日周五 下午9:53写道：
> > > > >
> > > > >> Hi Forward,
> > > > >>
> > > > >> Sorry for the late reply. I have went through the design doc and I
> > > think
> > > > >> it
> > > > >> is very nice.
> > > > >>
> > > > >> Here are my thoughts and suggestions:
> > > > >>
> > > > >> 0) I think support JSON functions in SQL is not complicated.
> Because
> > > > >> Calcite already supports the parser part and the runtime part.
> > > > >> We only need to integrate it in Flink and add good coverage
> > tests.
> > > > >> 1) However, I think we should also design the corresponding JSON
> > > > Functions
> > > > >> API for Table API which is very important.
> > > > >> I don't have a clear idea about how to support all the JSON
> > > Function
> > > > >> syntax in Table API. And this may need more discussions.
> > > > >> 2) IMO, it deserves a FLIP (especially for the Table API part).
> You
> > > can
> > > > >> follow the FLIP process [1] to start a FLIP proposal.
> > > > >> 3) I think we only need to implement it in blink planner as we are
> > > going
> > > > >> to
> > > > >> deprecate old planner.
> > > > >>So could you update the implementation section in the doc
> because
> > > the
> > > > >> implementation in blink planner should be different.
> > > > >> 4) It would be better to have an implementation plan to priority
> the
> > > > >> sub-tasks.
> > > > >> From my point of view, JSON_VALUE is the most important and
> > > > JSON_TABLE
> > > > >> gets the least priority.
> > > > >>
> > > > >> I also left some comments in the google doc.
> > > > >>
> > > > >> Hi @JingsongLee  ,
> > > > >>
> > > > >> I think we don't need to wait for FLIP-51. As we don't have a
> clear
> > > > >> progress of FLIP-51.
> > > > >> And as far as I know, it will add a few of PlannerExp

Re: Re: [DISCUSS] Support JSON functions in Flink SQL

[Forward Xu]

Hi Jingsong Lee :

Thank you very much, I need to apply for FLIP permission. Do I need to
create a FLIP for this?

Best,
Forward

Jingsong Li  于2019年12月2日周一 上午11:40写道：

> Hi Forward:
>
> Document looks good to me.
> I think you can just start doing this.
> They all work very independently, so I don't think there's any obvious
> blocking.
>
> Best,
> Jingsong Lee
>
> On Sat, Nov 30, 2019 at 10:59 AM Forward Xu 
> wrote:
>
> > Hi everyone, It's been a long time since I started this discussion. Do
> you
> > have anything to add and improve?
> > Best,
> > Forward
> >
> > Forward Xu  于2019年9月22日周日 下午6:30写道：
> >
> > > Hi Jack,
> > > Thank you very much for your reply, google doc I have updated, and some
> > of
> > > your questions I replied.
> > >  In addition, I want to apply for Flip permissions for this purpose.
> > >
> > > Best,
> > > Forward
> > >
> > > Jark Wu  于2019年9月20日周五 下午9:53写道：
> > >
> > >> Hi Forward,
> > >>
> > >> Sorry for the late reply. I have went through the design doc and I
> think
> > >> it
> > >> is very nice.
> > >>
> > >> Here are my thoughts and suggestions:
> > >>
> > >> 0) I think support JSON functions in SQL is not complicated. Because
> > >> Calcite already supports the parser part and the runtime part.
> > >> We only need to integrate it in Flink and add good coverage tests.
> > >> 1) However, I think we should also design the corresponding JSON
> > Functions
> > >> API for Table API which is very important.
> > >> I don't have a clear idea about how to support all the JSON
> Function
> > >> syntax in Table API. And this may need more discussions.
> > >> 2) IMO, it deserves a FLIP (especially for the Table API part). You
> can
> > >> follow the FLIP process [1] to start a FLIP proposal.
> > >> 3) I think we only need to implement it in blink planner as we are
> going
> > >> to
> > >> deprecate old planner.
> > >>So could you update the implementation section in the doc because
> the
> > >> implementation in blink planner should be different.
> > >> 4) It would be better to have an implementation plan to priority the
> > >> sub-tasks.
> > >> From my point of view, JSON_VALUE is the most important and
> > JSON_TABLE
> > >> gets the least priority.
> > >>
> > >> I also left some comments in the google doc.
> > >>
> > >> Hi @JingsongLee  ,
> > >>
> > >> I think we don't need to wait for FLIP-51. As we don't have a clear
> > >> progress of FLIP-51.
> > >> And as far as I know, it will add a few of PlannerExpressions which
> can
> > be
> > >> refactored easily during FLIP-51.
> > >>
> > >>
> > >> Cheers,
> > >> Jark
> > >>
> > >> [1]:
> > >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
> > >>
> > >>
> > >>
> > >> On Thu, 5 Sep 2019 at 19:29, vino yang  wrote:
> > >>
> > >> > +1 to have JSON functions in Flink SQL
> > >> >
> > >> > JingsongLee  于2019年9月5日周四
> 下午4:46写道：
> > >> >
> > >> > > +1
> > >> > > Nice document. I think it is easier to do after expression
> > >> reworking[1].
> > >> > > By the way, which planner do you want to start?
> > >> > >
> > >> > > [1]
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-51%3A+Rework+of+the+Expression+Design
> > >> > >
> > >> > > Best,
> > >> > > Jingsong Lee
> > >> > >
> > >> > >
> > >> > > --
> > >> > > From:TANG Wen-hui 
> > >> > > Send Time:2019年9月5日(星期四) 14:36
> > >> > > To:dev 
> > >> > > Subject:Re: Re: [DISCUSS] Support JSON functions in Flink SQL
> > >> > >
> > >> > > +1
> > >> > > I have done similar work before.
> > >> > > Looking forward to discussing this feature.
> > >>

Re: Re: [DISCUSS] Support JSON functions in Flink SQL

[Forward Xu]

Hi everyone, It's been a long time since I started this discussion. Do you
have anything to add and improve?
Best,
Forward

Forward Xu  于2019年9月22日周日 下午6:30写道：

> Hi Jack,
> Thank you very much for your reply, google doc I have updated, and some of
> your questions I replied.
>  In addition, I want to apply for Flip permissions for this purpose.
>
> Best,
> Forward
>
> Jark Wu  于2019年9月20日周五 下午9:53写道：
>
>> Hi Forward,
>>
>> Sorry for the late reply. I have went through the design doc and I think
>> it
>> is very nice.
>>
>> Here are my thoughts and suggestions:
>>
>> 0) I think support JSON functions in SQL is not complicated. Because
>> Calcite already supports the parser part and the runtime part.
>> We only need to integrate it in Flink and add good coverage tests.
>> 1) However, I think we should also design the corresponding JSON Functions
>> API for Table API which is very important.
>> I don't have a clear idea about how to support all the JSON Function
>> syntax in Table API. And this may need more discussions.
>> 2) IMO, it deserves a FLIP (especially for the Table API part). You can
>> follow the FLIP process [1] to start a FLIP proposal.
>> 3) I think we only need to implement it in blink planner as we are going
>> to
>> deprecate old planner.
>>So could you update the implementation section in the doc because the
>> implementation in blink planner should be different.
>> 4) It would be better to have an implementation plan to priority the
>> sub-tasks.
>> From my point of view, JSON_VALUE is the most important and JSON_TABLE
>> gets the least priority.
>>
>> I also left some comments in the google doc.
>>
>> Hi @JingsongLee  ,
>>
>> I think we don't need to wait for FLIP-51. As we don't have a clear
>> progress of FLIP-51.
>> And as far as I know, it will add a few of PlannerExpressions which can be
>> refactored easily during FLIP-51.
>>
>>
>> Cheers,
>> Jark
>>
>> [1]:
>>
>> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
>>
>>
>>
>> On Thu, 5 Sep 2019 at 19:29, vino yang  wrote:
>>
>> > +1 to have JSON functions in Flink SQL
>> >
>> > JingsongLee  于2019年9月5日周四 下午4:46写道：
>> >
>> > > +1
>> > > Nice document. I think it is easier to do after expression
>> reworking[1].
>> > > By the way, which planner do you want to start?
>> > >
>> > > [1]
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-51%3A+Rework+of+the+Expression+Design
>> > >
>> > > Best,
>> > > Jingsong Lee
>> > >
>> > >
>> > > --
>> > > From:TANG Wen-hui 
>> > > Send Time:2019年9月5日(星期四) 14:36
>> > > To:dev 
>> > > Subject:Re: Re: [DISCUSS] Support JSON functions in Flink SQL
>> > >
>> > > +1
>> > > I have done similar work before.
>> > > Looking forward to discussing this feature.
>> > >
>> > > Best
>> > > wenhui
>> > >
>> > >
>> > >
>> > > winifred.wenhui.t...@gmail.com
>> > >
>> > > From: Kurt Young
>> > > Date: 2019-09-05 14:00
>> > > To: dev
>> > > CC: Anyang Hu
>> > > Subject: Re: [DISCUSS] Support JSON functions in Flink SQL
>> > > +1 to add JSON support to Flink. We also see lots of requirements for
>> > JSON
>> > > related functions in our internal platform. Since these are already
>> SQL
>> > > standard, I think it's a good time to add them to Flink.
>> > >
>> > > Best,
>> > > Kurt
>> > >
>> > >
>> > > On Thu, Sep 5, 2019 at 10:37 AM Qi Luo  wrote:
>> > >
>> > > > We also see strong demands from our SQL users for JSON/Date related
>> > > > functions.
>> > > >
>> > > > Also +Anyang Hu 
>> > > >
>> > > > On Wed, Sep 4, 2019 at 9:51 PM Jark Wu  wrote:
>> > > >
>> > > > > Hi Forward,
>> > > > >
>> > > > > Thanks for bringing this discussion and preparing the nice design.
>> > > > > I think it's nice to have the JSON functions in the next release.
>> > > > > We have received som

Re: [ANNOUNCE] Jark Wu is now part of the Flink PMC

[Forward Xu]

Congratulations, Jark!


Best,

forward

Congxian Qiu  于2019年11月8日周五 下午9:27写道：

> Congratulations, Jark!
>
> Best,
> Congxian
>
>
> Leonard Xu  于2019年11月8日周五 下午8:40写道：
>
> > Congratulations, Jark.
> > Thanks for your contribution and help.
> >
> > Best,
> > Leonard Xu
> >
> > > On 2019年11月8日, at 下午6:37, Yun Gao 
> wrote:
> > >
> > > Congratulations Jark！
> > >
> > > Best,
> > > Yun
> > >
> > >
> > > --
> > > From:wenlong.lwl 
> > > Send Time:2019 Nov. 8 (Fri.) 18:31
> > > To:dev 
> > > Subject:Re: [ANNOUNCE] Jark Wu is now part of the Flink PMC
> > >
> > > Congratulations Jark, well deserved!
> > >
> > >
> > > Best,
> > > Wenlong Lyu
> > >
> > > On Fri, 8 Nov 2019 at 18:22, tison  wrote:
> > >
> > >> Congrats Jark!
> > >>
> > >> Best,
> > >> tison.
> > >>
> > >>
> > >> Jingsong Li  于2019年11月8日周五 下午6:08写道：
> > >>
> > >>> Congratulations to Jark.
> > >>> Jark has really contributed a lot to the table layer with a long
> time.
> > >> Well
> > >>> deserved.
> > >>>
> > >>> Best,
> > >>> Jingsong Lee
> > >>>
> > >>> On Fri, Nov 8, 2019 at 6:05 PM Yu Li  wrote:
> > >>>
> >  Congratulations Jark! Well deserved!
> > 
> >  Best Regards,
> >  Yu
> > 
> > 
> >  On Fri, 8 Nov 2019 at 17:55, OpenInx  wrote:
> > 
> > > Congrats Jark ! Well deserve.
> > >
> > > On Fri, Nov 8, 2019 at 5:53 PM Paul Lam 
> > >> wrote:
> > >
> > >> Congrats Jark!
> > >>
> > >> Best,
> > >> Paul Lam
> > >>
> > >>> 在 2019年11月8日，17:51，jincheng sun  写道：
> > >>>
> > >>> Hi all,
> > >>>
> > >>> On behalf of the Flink PMC, I'm happy to announce that Jark Wu is
> > >>> now
> > >>> part of the Apache Flink Project Management Committee (PMC).
> > >>>
> > >>> Jark has been a committer since February 2017. He has been very
> >  active
> > > on
> > >>> Flink's Table API / SQL component, as well as frequently helping
> > >>> manage/verify/vote releases. He has been writing many blogs about
> > > Flink,
> > >>> also driving the translation work of Flink website and
> > >>> documentation.
> > > He
> > >> is
> > >>> very active in China community as he gives talks about Flink at
> > >>> many
> > >> events
> > >>> in China.
> > >>>
> > >>> Congratulations & Welcome Jark!
> > >>>
> > >>> Best,
> > >>> Jincheng (on behalf of the Flink PMC)
> > >>
> > >>
> > >
> > 
> > >>>
> > >>>
> > >>> --
> > >>> Best, Jingsong Lee
> > >>>
> > >>
> >
> >
>

Confluence permission for FLIP creation

[Forward Xu]

Hi devs,

I'd like to create a page about the Support SQL 2016-2017 JSON functions in
Flink SQL

FLIP. Could you grant me Confluence permission for FLIP creation?

My Confluence ID is forwardxu.

Best,
Forward.

Re: Re: [DISCUSS] Support JSON functions in Flink SQL

[Forward Xu]

Hi Jack,
Thank you very much for your reply, google doc I have updated, and some of
your questions I replied.
 In addition, I want to apply for Flip permissions for this purpose.

Best,
Forward

Jark Wu  于2019年9月20日周五 下午9:53写道：

> Hi Forward,
>
> Sorry for the late reply. I have went through the design doc and I think it
> is very nice.
>
> Here are my thoughts and suggestions:
>
> 0) I think support JSON functions in SQL is not complicated. Because
> Calcite already supports the parser part and the runtime part.
> We only need to integrate it in Flink and add good coverage tests.
> 1) However, I think we should also design the corresponding JSON Functions
> API for Table API which is very important.
> I don't have a clear idea about how to support all the JSON Function
> syntax in Table API. And this may need more discussions.
> 2) IMO, it deserves a FLIP (especially for the Table API part). You can
> follow the FLIP process [1] to start a FLIP proposal.
> 3) I think we only need to implement it in blink planner as we are going to
> deprecate old planner.
>So could you update the implementation section in the doc because the
> implementation in blink planner should be different.
> 4) It would be better to have an implementation plan to priority the
> sub-tasks.
> From my point of view, JSON_VALUE is the most important and JSON_TABLE
> gets the least priority.
>
> I also left some comments in the google doc.
>
> Hi @JingsongLee  ,
>
> I think we don't need to wait for FLIP-51. As we don't have a clear
> progress of FLIP-51.
> And as far as I know, it will add a few of PlannerExpressions which can be
> refactored easily during FLIP-51.
>
>
> Cheers,
> Jark
>
> [1]:
>
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
>
>
>
> On Thu, 5 Sep 2019 at 19:29, vino yang  wrote:
>
> > +1 to have JSON functions in Flink SQL
> >
> > JingsongLee  于2019年9月5日周四 下午4:46写道：
> >
> > > +1
> > > Nice document. I think it is easier to do after expression
> reworking[1].
> > > By the way, which planner do you want to start?
> > >
> > > [1]
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-51%3A+Rework+of+the+Expression+Design
> > >
> > > Best,
> > > Jingsong Lee
> > >
> > >
> > > --
> > > From:TANG Wen-hui 
> > > Send Time:2019年9月5日(星期四) 14:36
> > > To:dev 
> > > Subject:Re: Re: [DISCUSS] Support JSON functions in Flink SQL
> > >
> > > +1
> > > I have done similar work before.
> > > Looking forward to discussing this feature.
> > >
> > > Best
> > > wenhui
> > >
> > >
> > >
> > > winifred.wenhui.t...@gmail.com
> > >
> > > From: Kurt Young
> > > Date: 2019-09-05 14:00
> > > To: dev
> > > CC: Anyang Hu
> > > Subject: Re: [DISCUSS] Support JSON functions in Flink SQL
> > > +1 to add JSON support to Flink. We also see lots of requirements for
> > JSON
> > > related functions in our internal platform. Since these are already SQL
> > > standard, I think it's a good time to add them to Flink.
> > >
> > > Best,
> > > Kurt
> > >
> > >
> > > On Thu, Sep 5, 2019 at 10:37 AM Qi Luo  wrote:
> > >
> > > > We also see strong demands from our SQL users for JSON/Date related
> > > > functions.
> > > >
> > > > Also +Anyang Hu 
> > > >
> > > > On Wed, Sep 4, 2019 at 9:51 PM Jark Wu  wrote:
> > > >
> > > > > Hi Forward,
> > > > >
> > > > > Thanks for bringing this discussion and preparing the nice design.
> > > > > I think it's nice to have the JSON functions in the next release.
> > > > > We have received some requirements for this feature.
> > > > >
> > > > > I can help to shepherd this JSON functions effort and will leave
> > > comments
> > > > >  in the design doc in the next days.
> > > > >
> > > > > Hi Danny,
> > > > >
> > > > > The new introduced JSON functions are from SQL:2016, not from
> MySQL.
> > > > > So there no JSON type is needed. According to the SQL:2016, the
> > > > > representation of JSON data can be "character string" which is also
> > > > > the current implementation in Calcite[1].
> > > > >
> > > > > Best,
> > > > > Jark
> > > > >
> > > > >
> > > > > [1]: https://calcite.apache.org/docs/reference.html#json-functions
> > > > >
> > > > >
> > > > > On Wed, 4 Sep 2019 at 21:22, Xu Forward 
> > > wrote:
> > > > >
> > > > > > hi Danny Chan ,Thank you very much for your reply, your help can
> > help
> > > > me
> > > > > > further improve this discussion.
> > > > > > Best
> > > > > > forward
> > > > > >
> > > > > > Danny Chan  于2019年9月4日周三 下午8:50写道：
> > > > > >
> > > > > > > Thanks Xu Forward for bring up this topic, I think the JSON
> > > functions
> > > > > are
> > > > > > > very useful especially for those MySQL users.
> > > > > > >
> > > > > > > I saw that you have done some work within the Apache Calcite,
> > > that’s
> > > > a
> > > > > > > good start, but this is one concern from me, Flink doesn’t
> > support
> > > > JSON
> > > > > > > type internal, so how to represent a JSON object in Flink
> maybe a

Re: How stable is FlinkSQL.

[Forward Xu]

Hi Srikanth,


Here are some past developer profiles you can view.

https://ververica.cn/developers-resources/

Many of them are cases of Flink-SQL.


Best,

Forward

srikanth flink  于2019年9月16日周一 下午9:39写道：

> Hi Kurt,
>
> thanks for quick response. Is the email user@ml?
>
> Regards
> Srikanth
>
>
> On Mon, Sep 16, 2019 at 1:31 PM Kurt Young  wrote:
>
> > Hi Srikanth,
> >
> > AFAIK, there are quite some companies already using Flink streaming
> > SQL to back their production systems, like realtime data warehouse. If
> > you met some issues when trying streaming sql, I would suggest you to
> > send the problem to user@ml, where you can receive some helps.
> >
> > Best,
> > Kurt
> >
> >
> > On Mon, Sep 16, 2019 at 2:44 PM srikanth flink 
> > wrote:
> >
> > > Hi Stephan,
> > >
> > > - I'm using streaming SQL
> > > - 1.9 version as recommended by Flink to have a updated stable version.
> > >
> > > Thanks
> > > Srikanth
> > >
> > > On Fri, Sep 13, 2019 at 8:30 PM Stephan Ewen  wrote:
> > >
> > > > Can you share some more details?
> > > >
> > > >   - are you running batch SQL or streaming SQL
> > > >   - are you running the original Flink SQL engine or the new Blink
> SQL
> > > > engine (since 1.9)
> > > >
> > > > Best,
> > > > Stephan
> > > >
> > > >
> > > > On Fri, Sep 13, 2019 at 3:24 PM srikanth flink  >
> > > > wrote:
> > > >
> > > > > Hi there,
> > > > >
> > > > > I'm trying to get some hands on with FlinkSQL and take to
> production,
> > > if
> > > > > works. Would like to know if someone deployed FlinkSQL in
> production?
> > > > >
> > > > > I'm facing issues while running FlinkSQL queries.
> > > > >
> > > > > Thanks
> > > > > Srikanth
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Forward Xu]

Congratulations Zili Chen!


Best,

Forward

Till Rohrmann  于2019年9月11日周三 下午5:23写道：

> Hi everyone,
>
> I'm very happy to announce that Zili Chen (some of you might also know him
> as Tison Kun) accepted the offer of the Flink PMC to become a committer of
> the Flink project.
>
> Zili Chen has been an active community member for almost 16 months now. He
> helped pushing the Flip-6 effort over the finish line, ported a lot of
> legacy code tests, removed a good part of the legacy code, contributed
> numerous fixes, is involved in the Flink's client API refactoring, drives
> the refactoring of Flink's HighAvailabilityServices and much more. Zili
> Chen also helped the community by PR reviews, reporting Flink issues,
> answering user mails and being very active on the dev mailing list.
>
> Congratulations Zili Chen!
>
> Best, Till
> (on behalf of the Flink PMC)
>

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

[Forward Xu]

Congratulations Kloudas!


Best,

Forward

Dawid Wysakowicz  于2019年9月6日周五 下午10:36写道：

> Congratulations Klou!
>
> Best,
>
> Dawid
>
> On 06/09/2019 14:55, Fabian Hueske wrote:
> > Hi everyone,
> >
> > I'm very happy to announce that Kostas Kloudas is joining the Flink PMC.
> > Kostas is contributing to Flink for many years and puts lots of effort in
> > helping our users and growing the Flink community.
> >
> > Please join me in congratulating Kostas!
> >
> > Cheers,
> > Fabian
> >
>
>

Re: [DISCUSS] FLIP-66: Support time attribute in SQL DDL

[Forward Xu]

Thanks Jark for this topic, This will be very useful.


Best,

ForwardXu

Danny Chan  于2019年9月6日周五 上午11:26写道：

> Thanks Jark for bring up this topic, this is definitely an import feature
> for the SQL, especially the DDL users.
>
> I would spend some time to review this design doc, really thanks.
>
> Best,
> Danny Chan
> 在 2019年9月6日 +0800 AM11:19，Jark Wu ，写道：
> > Hi everyone,
> >
> > I would like to start discussion about how to support time attribute in
> SQL
> > DDL.
> > In Flink 1.9, we already introduced a basic SQL DDL to create a table.
> > However, it doesn't support to define time attributes. This makes users
> > can't
> > apply window operations on the tables created by DDL which is a bad
> > experience.
> >
> > In FLIP-66, we propose a syntax for watermark to define rowtime attribute
> > and propose to use computed column syntax to define proctime attribute.
> > But computed column is another big topic and should deserve a separate
> > FLIP.
> > If we have a consensus on the computed column approach, we will start
> > computed column FLIP soon.
> >
> > FLIP-66:
> >
> https://docs.google.com/document/d/1-SecocBqzUh7zY6HBYcfMlG_0z-JAcuZkCvsmN3LrOw/edit#
> >
> > Thanks for any feedback!
> >
> > Best,
> > Jark
>