Hi Sijie,
Yes, we will have to support existing old connectors and new connectors in
parallel for a while. We have to take that maintenance overhead because
existing connectors have been used by the users for a long time. I guess It
may take at least a year for us to fully remove the old
Yadong Xie created FLINK-14138:
--
Summary: Show Pending Slots in Job Detail
Key: FLINK-14138
URL: https://issues.apache.org/jira/browse/FLINK-14138
Project: Flink
Issue Type: Improvement
Yadong Xie created FLINK-14137:
--
Summary: Show Attempt History in Vertex SubTask
Key: FLINK-14137
URL: https://issues.apache.org/jira/browse/FLINK-14137
Project: Flink
Issue Type: Improvement
I'm not sure if I understand the implementation plan you suggested
correctly. To my understanding, it seems that all the steps except for step
5 have to happen in strict order.
- Profiles to be used in step 2 is reported with step 1.
- SlotProfile in TaskExecutorGateway#requestSlot in step
Thanks Steven for the feedback!
Could you share more information about the metrics you add in you
customized restart strategy?
Thanks,
Zhu Zhu
Steven Wu 于2019年9月20日周五 上午7:11写道：
> We do use config like "restart-strategy:
> org.foobar.MyRestartStrategyFactoryFactory". Mainly to add additional
>
Yadong Xie created FLINK-14136:
--
Summary: Operator Topology and Metrics Inside Vertex
Key: FLINK-14136
URL: https://issues.apache.org/jira/browse/FLINK-14136
Project: Flink
Issue Type:
Jingsong Lee created FLINK-14135:
Summary: Introduce vectorized orc InputFormat for blink runtime
Key: FLINK-14135
URL: https://issues.apache.org/jira/browse/FLINK-14135
Project: Flink
Thanks everyone here. Sorry for jumping into the discussion here.
I am not very familiar about the deprecation process in Flink. If I
misunderstood the process, please fix me.
As far as I understand, FLIP-27 is introducing a new unified API for
connectors. After it introduces the new API
and
Jingsong Lee created FLINK-14134:
Summary: Introduce LimitableTableSource to optimize limit
Key: FLINK-14134
URL: https://issues.apache.org/jira/browse/FLINK-14134
Project: Flink
Issue Type:
Jingsong Lee created FLINK-14133:
Summary: Improve batch sql and hive integrate performance
Key: FLINK-14133
URL: https://issues.apache.org/jira/browse/FLINK-14133
Project: Flink
Issue Type:
AFAIK, grouping sets has already been supported for streaming in blink planner.
You could check FLINK-12192 for details.
Regards,
Dian
> 在 2019年9月10日，下午6:51，刘建刚 写道：
>
> I want to implement grouping set in stream. I am new to flink sql. I
> want to find a example to teach me how to self
We do use config like "restart-strategy:
org.foobar.MyRestartStrategyFactoryFactory". Mainly to add additional
metrics than the Flink provided ones.
On Thu, Sep 19, 2019 at 4:50 AM Zhu Zhu wrote:
> Thanks everyone for the input.
>
> The RestartStrategy customization is not recognized as a
Thanks everyone for your feedback. I've converted it to a FLIP wiki [1].
Please take another look. If there's no more concerns, I'd like to start a
voting thread for it.
Thanks
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-68%3A+Extend+Core+Table+System+with+Modular+Plugins
On
Bowen Li created FLINK-14132:
Summary: Extend core table system with modular plugins
Key: FLINK-14132
URL: https://issues.apache.org/jira/browse/FLINK-14132
Project: Flink
Issue Type: New
Another reason I prefer "CREATE TEMPORARY BUILTIN FUNCTION" over "ALTER
BUILTIN FUNCTION xxx TEMPORARILY" is - what if users want to drop the
temporary built-in function in the same session? With the former one, they
can run something like "DROP TEMPORARY BUILTIN FUNCTION"; With the latter
one,
Thanks for the explanation, Stephan. I have a few questions / thoughts.
So that means we will remove the old connector without a major version
bump, is that correct?
I am not 100% sure if mixing 1.10 connectors with 1.11 connectors will
always work because we saw some dependency class collisions
Hi,
Thanks everyone for your votes. I summarized the result as following:
#1：3 (+1), 1 (0), 4(-1) - net: -1
#2：4(0), 2 (+1), 1(+0.5) - net: +2.5
Dawid -1/0 depending on keyword
#3：2(+1), 3(-1), 3(0) - net: -1
Given the result, I'd like to change my vote for #2 from 0 to +1,
Hi everyone,
Thanks all for the valuable suggestions and feedbacks so far.
Before starting the vote, I would like to summarize the proposed DDL syntax
in the mailing list.
## Rowtime Attribute (Watermark Syntax)
CREATE TABLE table_name (
WATERMARK FOR AS
) WITH (
...
)
It marks an
Zhu Zhu created FLINK-14131:
---
Summary: Support choosing new version failover strategy via
configuration
Key: FLINK-14131
URL: https://issues.apache.org/jira/browse/FLINK-14131
Project: Flink
Thanks for your suggestions and information.
I'll therefore reopen the corresponding pull request.
Best,
tison.
Till Rohrmann 于2019年9月18日周三 下午10:55写道：
> No reason to keep the separation. The NewClusterClient interface was only
> introduced to add new methods and not having to implement them
Aljoscha Krettek created FLINK-14130:
Summary: Remove ClusterClient.run() methods
Key: FLINK-14130
URL: https://issues.apache.org/jira/browse/FLINK-14130
Project: Flink
Issue Type:
Hi,
There are some new valuable suggestions and comments in the design
documentation since the vote started.
I will update the documentation and summarize it in the [DISCUSS] thread. I
hereby cancel the vote.
Will start a new vote once we get a general consensus on the changes.
Thanks,
Jark
On
Rui Li created FLINK-14129:
--
Summary: HiveTableSource should implement ProjectableTableSource
Key: FLINK-14129
URL: https://issues.apache.org/jira/browse/FLINK-14129
Project: Flink
Issue Type:
Zhu Zhu created FLINK-14128:
---
Summary: Remove the description of restart strategy customization
Key: FLINK-14128
URL: https://issues.apache.org/jira/browse/FLINK-14128
Project: Flink
Issue Type:
Hi Stephen,
that sounds like a kind of option. I'm not sure if this would work on
JDK11, though. JDK11 where I found this issue in first place, because on
JDK < 9 I can inject the generated file into application class loader
(which is URLClassLoader). This class loader changed in JDK9 and
I think besides of point 1. and 3. there are no dependencies between the RM
and TM side changes. Also, I'm not sure whether it makes sense to split the
slot manager changes up into the proposed steps 5, 6 and 7.
I would highly recommend to not add too much duplicate logic/separate code
paths
Thanks everyone for the input.
The RestartStrategy customization is not recognized as a public interface
as it is not explicitly documented.
As it is not used from the feedbacks of this survey, I'll conclude that we
do not need to support customized RestartStrategy for the new scheduler in
Flink
Hi Terry,
I gave you permissions.
Thanks, Fabian
Am Do., 19. Sept. 2019 um 04:09 Uhr schrieb Terry Wang :
> Hi all,
>
> As communicated in an email thread, I'm proposing Flink SQL ddl
> enhancement. I have a draft design doc that I'd like to convert it to a
> FLIP. Thus, it would be great if
My take would be the following:
- If we merge the connector now and replace it with a FLIP-27 version
before the 1.10 release, then we need no deprecation process
- If we don't manage to replace it with a FLIP-27 version before the 1.10
release, than it is good that we have the other version,
Hi Jan!
You can also add an additional class path URL to take classes from. Does
that help?
Is there a directly where the generated class files go, so that a
local file URL could reference them and add them to the user code class
loader?
Best,
Stephan
On Tue, Sep 10, 2019 at 10:28 AM Till
Yadong Xie created FLINK-14127:
--
Summary: Better BackPressure Detection in WebUI
Key: FLINK-14127
URL: https://issues.apache.org/jira/browse/FLINK-14127
Project: Flink
Issue Type: Improvement
I agree, it's very similar from the implementation point of view and the
implications.
IMO, the difference is mostly on the mental model for the user.
Instead of having a special class of temporary functions that have
precedence over builtin functions it suggests to temporarily change
built-in
Thanks for the comments, Till.
- Agree on removing SlotID.
- Regarding the implementation plan, it is true that we can possibly reduce
codes separated by the feature option. But I think to do that we need to
introduce more dependencies between implementation steps. With the current
plan, we can
Hi Fabian,
I think it's almost the same with #2 with different keyword:
CREATE TEMPORARY BUILTIN FUNCTION xxx
Best,
Kurt
On Thu, Sep 19, 2019 at 5:50 PM Fabian Hueske wrote:
> Hi,
>
> I thought about it a bit more and think that there is some good value in my
> last proposal.
>
> A lot of
Hi,
I thought about it a bit more and think that there is some good value in my
last proposal.
A lot of complexity comes from the fact that we want to allow overriding
built-in functions which are differently addressed as other functions (and
db objects).
We could just have "CREATE TEMPORARY
My gut feeling is that having a CI that only runs on a specific command
will not help too much.
What about going with nightly builds then? We could set up the ARM CI the
same way as the Travis CI nightly builds (cron builds). They report build
failures to "bui...@flink.apache.org".
Maybe Chesnay
Thanks!
Best,
tison.
Till Rohrmann 于2019年9月19日周四 下午5:34写道：
> Granted. You should now be able to create pages Tison.
>
> Cheers,
> Till
>
> On Thu, Sep 19, 2019 at 11:01 AM Zili Chen wrote:
>
> > Hi devs,
> >
> > I'd like to create a page about the ongoing JobClient FLIP. Could you
> > grant
Hi,
You should have the necessary permissions.
Let me know if something doesn't work.
Cheers, Fabian
Am Do., 19. Sept. 2019 um 10:56 Uhr schrieb Yijie Shen <
henry.yijies...@gmail.com>:
> Hi Fabian,
>
> It's yijieshen
>
> Thanks for your help!
>
> On Thu, Sep 19, 2019 at 4:52 PM Fabian
wangxiyuan created FLINK-14126:
--
Summary: Elasticsearch Xpack Machine Learning doesn't support ARM
Key: FLINK-14126
URL: https://issues.apache.org/jira/browse/FLINK-14126
Project: Flink
Issue
Granted. You should now be able to create pages Tison.
Cheers,
Till
On Thu, Sep 19, 2019 at 11:01 AM Zili Chen wrote:
> Hi devs,
>
> I'd like to create a page about the ongoing JobClient FLIP. Could you
> grant me Confluence permission for FLIP creation?
>
> My Confluence ID is tison.
>
>
Hi everyone,
I thought again about option #1 and something that I don't like is that the
resolved address of xyz is different in "CREATE FUNCTION xyz" and "CREATE
TEMPORARY FUNCTION xyz".
IMO, adding the keyword "TEMPORARY" should only change the lifecycle of the
function, but not where it is
After reading Kurt’s reasoning I’m bumping my vote for #2 from -1 to +0, or
even +0.5, so my final vote is:
-1 for #1
+0.5 for #2
+1 for #3
Re confusion about “system_db”. I think quite a lot of DBs are storing some
meta tables in some system and often hidden db/schema, so I don’t think that
Hi Xintong,
thanks for starting the vote. The general plan looks good. Hence +1 from my
side. I still have some minor comments one could think about:
* As we no longer have predetermined slots on the TaskExecutor, I think we
can get rid of the SlotID. Instead, an allocated slot will be
Hi Stephan,
Thanks for the clarification. I completely agree with you and Thomas on the
process of adding connectors to Flink repo. However, I am wondering what is
the deprecation process? Given the main concern here was that we may have
to maintain two Pulsar connector code bases until the old
Yadong Xie created FLINK-14125:
--
Summary: Display memory and CPU usage in the overview page
Key: FLINK-14125
URL: https://issues.apache.org/jira/browse/FLINK-14125
Project: Flink
Issue Type:
I know Hive and Spark can shadow built-in functions by temporary function.
Mysql, Oracle, Sql server can not shadow.
User can use full names to access functions instead of shadowing.
So I think it is a completely new thing, and the direct way to deal with new
things is to add new grammar. So,
+1
Hi devs,
I'd like to create a page about the ongoing JobClient FLIP. Could you
grant me Confluence permission for FLIP creation?
My Confluence ID is tison.
Best,
tison.
Hi Fabian,
It's yijieshen
Thanks for your help!
On Thu, Sep 19, 2019 at 4:52 PM Fabian Hueske wrote:
>
> Hi Yijie,
>
> I can give you permission if you tell me your Confluence user name.
>
> Thanks, Fabian
>
> Am Do., 19. Sept. 2019 um 10:49 Uhr schrieb Yijie Shen <
>
Hi Yijie,
I can give you permission if you tell me your Confluence user name.
Thanks, Fabian
Am Do., 19. Sept. 2019 um 10:49 Uhr schrieb Yijie Shen <
henry.yijies...@gmail.com>:
> > > > Could you please follow the FLIP process to start a new FLIP
> [DISCUSSION]
> > > > thread in the mailing
> > > Could you please follow the FLIP process to start a new FLIP [DISCUSSION]
> > > thread in the mailing list?
It seems that I don't have permission to create pages in FLIP.
> > > > Thanks for sharing the pulsar FLIP.
> > > > Would you mind enabling comments/suggestions on the google doc
And let me make my vote complete:
-1 for #1
+1 for #2 with different keyword
-0 for #3
Best,
Kurt
On Thu, Sep 19, 2019 at 4:40 PM Kurt Young wrote:
> Looks like I'm the only person who is willing to +1 to #2 for now :-)
> But I would suggest to change the keyword from GLOBAL to
> something
Looks like I'm the only person who is willing to +1 to #2 for now :-)
But I would suggest to change the keyword from GLOBAL to
something like BUILTIN.
I think #2 and #3 are almost the same proposal, just with different
format to indicate whether it want to override built-in functions.
My biggest
YufeiLiu created FLINK-14124:
Summary: potential memory leak in netty server
Key: FLINK-14124
URL: https://issues.apache.org/jira/browse/FLINK-14124
Project: Flink
Issue Type: Improvement
+1 to what Stephan (and Thomas) said.
Am Do., 19. Sept. 2019 um 09:54 Uhr schrieb Stephan Ewen :
> Some quick thoughts on the connector contribution process. I basically
> reiterate here what Thomas mentioned in another thread about the Kinesis
> connector.
>
> For connectors, we should favor a
Some quick thoughts on the connector contribution process. I basically
reiterate here what Thomas mentioned in another thread about the Kinesis
connector.
For connectors, we should favor a low-overhead contribution process, and
accept user code and changes more readily than in the core system.
Hi Xintong,
Thanks for starting the vote, +1 from my side.
Best,
Andrey
On Tue, Sep 17, 2019 at 4:26 PM Xintong Song wrote:
> Hi all,
>
> I would like to start the vote for FLIP-56 [1], on which a consensus is
> reached in this discussion thread [2].
>
> The vote will be open for at least 72
Hi,
if there is an easy way to make it also work with Java 1.8.0_77 I guess we
could change it. That way we would make the life of our users easier.
The solution proposed by JDK-8054569 seems quite simple. The only downside
I see is that it could easily fell victim of a future refactoring/clean
IIUC it's good to see that both serializable (tables description from DDL)
and unserializable (tables with DataStream underneath) tables are treated
unify with CatalogTable.
Can I also assume functions that either come from a function class (from
DDL)
or function objects (newed by user) will also
Thanks Dawid for the design doc.
In general, I’m +1 to the FLIP.
+1 to the single-string and parse way to express object path.
+1 to deprecate registerTableSink & registerTableSource.
But I would suggest to provide an easy way to register a custom source/sink
before we drop them (this is
Hi,
It is a quite long discussion to follow and I hope I didn’t misunderstand
anything. From the proposals presented by Xuefu I would vote:
-1 for #1 and #2
+1 for #3
Besides #3 being IMO more general and more consistent, having qualified names
(#3) would help/make easier for someone to use
Hey all,
Recently, I used flink to do secondary development, when compile flink
master(up-to-date) by using Java 1.8.0_77, got errors as follow:
compile (default-compile) on project flink-table-api-java: Compilation
failure
liupengcheng created FLINK-14123:
Summary: Change taskmanager.memory.fraction default value to 0.6
Key: FLINK-14123
URL: https://issues.apache.org/jira/browse/FLINK-14123
Project: Flink
62 matches
Mail list logo
