Hi everyone,
I completely reworked FLIP-107. It now covers the full story how to read
and write metadata from/to connectors and formats. It considers all of
the latest FLIPs, namely FLIP-95, FLIP-132 and FLIP-122. It introduces
the concept of PERSISTED computed columns and leaves out
Hi Danny,
"if ChangelogMode.INSERT is the default, existing pipelines should be
compatible"
It is not about changelog mode compatibility, it is about the type
compatibility. The renaming to `toInsertStream` is only to have a mean
of dealing with data type inconsistencies that could break
Best,
Danny Chan
在 2020年9月2日 +0800 PM4:19,Timo Walther ,写道:
Hi everyone
thanks for your feedback. It's a lot of content that needs to be
digested. I will update the FLIP shortly to incorporate some of the
feedback already. But let me respond to some topics first:
"not deprecate these API&
Timo Walther created FLINK-19127:
Summary: Provide a replacement of
StreamExecutionEnvironment.createRemoteEnvironment for TableEnvironment
Key: FLINK-19127
URL: https://issues.apache.org/jira/browse/FLINK-19127
Hi Jingsong,
I haven't looked at your proposal but I think it make sense to have a
separate FLIP for the parititioning topic. I'm currently working on an
update to FLIP-107 and would suggest to remove the paritioning topic
there. FLIP-107 will only focus on accessing metadata and expressing
y. In the SQL world, no projection function outputs type of
time-attribute, we better still put the time-attributes in the scope of the
table metadata.
Best,
Danny Chan
在 2020年8月19日 +0800 PM4:22,Timo Walther ,写道:
Hi everyone,
I would like to propose a FLIP that aims to resolve the remaining
shortc
ironment#execute.
2. @Timo What is the interaction between Row setters from the different
modes? What happens if the user calls both in different order. E.g.
row.setField(0, "ABC");
row.setField("f0", "ABC"); // is this a valid call ?
or
row.setField("f0&qu
ames` parameter is
enough and more handy than Map ?
- Currently, the fieldNames member variable is mutable, is it on purpose?
Can we make it immutable? For example, only accept from the constructor.
- Why do we accept a nullable `fieldNames`?
7. "a Row has two modes represented by an inter
Hi Wei,
is `reset_accumulator` still necessary? We dropped it recently in the
Java API because it was not used anymore by the planner.
Regards,
Timo
On 31.08.20 15:00, Wei Zhong wrote:
Hi Jincheng & Xingbo,
Thanks for your suggestions.
I agree that we should keep the user interface
+1
Thanks for removing legacy.
Regards,
Timo
On 28.08.20 11:55, David Anderson wrote:
+1
David
On Fri, Aug 28, 2020 at 9:41 AM Dawid Wysakowicz
wrote:
Hi all,
I would like to start a vote for removing deprecated, but Public(Evolving)
methods in the upcoming 1.12 release:
-
. The statements in a
statement
set are jointly optimized and executed as a single Flink job.
Maybe if you add this to the FLIP it will help other readers as well.
Best,
David
On Wed, Aug 19, 2020 at 10:22 AM Timo Walther wrote:
Hi everyone,
I would like to propose a FLIP that ai
Hi everyone,
I would like to propose a FLIP that aims to resolve the remaining
shortcomings in the Table API:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-136%3A++Improve+interoperability+between+DataStream+and+Table+API
The Table API has received many new features over the last
Timo Walther created FLINK-18936:
Summary: Update documentation about user-defined aggregate
functions
Key: FLINK-18936
URL: https://issues.apache.org/jira/browse/FLINK-18936
Project: Flink
Timo Walther created FLINK-18901:
Summary: Use new type inference for SQL DDL of aggregate functions
Key: FLINK-18901
URL: https://issues.apache.org/jira/browse/FLINK-18901
Project: Flink
+1 (binding)
I went through the commit diff and changed files of this release. Could
not spot anything suspicious.
Regards,
Timo
On 11.08.20 14:47, Zhu Zhu wrote:
Hi everyone,
Please review and vote on the release candidate #1 for the version 1.10.2,
as follows:
[ ] +1, Approve the release
Timo Walther created FLINK-18890:
Summary: Update AsyncTableFunctions to the new type system
Key: FLINK-18890
URL: https://issues.apache.org/jira/browse/FLINK-18890
Project: Flink
Issue Type
Timo Walther created FLINK-18877:
Summary: Faulty BinarySection.equals()
Key: FLINK-18877
URL: https://issues.apache.org/jira/browse/FLINK-18877
Project: Flink
Issue Type: Bug
Hi Xuannan,
sorry for joining the discussion so late. I agree that this is a very
nice and useful feature. However, the impact it has to many components
in the stack requires more discussion in my opinion.
1) Separation of concerns:
The current design seems to mix different layers. We should
Timo Walther created FLINK-18840:
Summary: Support StatementSet with DataStream API
Key: FLINK-18840
URL: https://issues.apache.org/jira/browse/FLINK-18840
Project: Flink
Issue Type: New
Timo Walther created FLINK-18809:
Summary: Update internal aggregate functions to new type system
Key: FLINK-18809
URL: https://issues.apache.org/jira/browse/FLINK-18809
Project: Flink
Issue
Hi Leonard,
sorry for jumping into the discussion so late. But I have two questions:
1) Naming: Is operation time a good term for this concept? If I read
"The operation time is the time when the changes happened in system." or
"The system time of DML execution in database", why don't we call
+1
Thanks for driving this Jark.
Regards,
Timo
On 24.07.20 12:42, Jark Wu wrote:
Hi all,
I would like to start the vote for FLIP-129 [1], which is discussed and
reached consensus in the discussion thread [2].
The vote will be open until 27th July (72h), unless there is an objection
or not
Timo Walther created FLINK-18703:
Summary: Use new data structure converters when legacy types are
not present
Key: FLINK-18703
URL: https://issues.apache.org/jira/browse/FLINK-18703
Project: Flink
Timo Walther created FLINK-18699:
Summary: Allow variables for column names in Scala Table API
Key: FLINK-18699
URL: https://issues.apache.org/jira/browse/FLINK-18699
Project: Flink
Issue
ConfigOption`
I’m slightly -1 for this, ConfigOption may not work because the key for
format configOption has not format prefix eg: FAIL_ON_MISSING_FIELD of
json, we need “json.fail-on-missing-field” rather than
“fail-on-missing-field”.
public static final ConfigOption FAIL_ON_MISSING_FIELD =
ConfigOptions
I agree with Dian. We can release a 1.11.2 shortly afterwards. Only
regressions compared to 1.11.0 should block this vote.
Regards,
Timo
On 17.07.20 10:48, Dian Fu wrote:
Generally, I tend to continue the vote if this is not a blocking issue for the
following reasons:
- As discussed in the
Timo Walther created FLINK-18628:
Summary: Invalid error message for overloaded methods with same
parameter name
Key: FLINK-18628
URL: https://issues.apache.org/jira/browse/FLINK-18628
Project: Flink
Hi Jark,
thanks for working on this issue. It is time to fix this last part of
inconsistency in the API. I also like the core parts of the FLIP, esp.
that TableDescriptor is one entity that can be passed to different
methods. Here is some feedback from my side:
1) +1 for just `column(...)`
Timo Walther created FLINK-18586:
Summary: Simplify the creation of explicit structured types
Key: FLINK-18586
URL: https://issues.apache.org/jira/browse/FLINK-18586
Project: Flink
Issue
Timo Walther created FLINK-18553:
Summary: Update Table API set operations to the new type system
Key: FLINK-18553
URL: https://issues.apache.org/jira/browse/FLINK-18553
Project: Flink
Issue
Timo Walther created FLINK-18537:
Summary: Remove RowDataTypeInfo
Key: FLINK-18537
URL: https://issues.apache.org/jira/browse/FLINK-18537
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-18528:
Summary: Update UNNEST to new type system
Key: FLINK-18528
URL: https://issues.apache.org/jira/browse/FLINK-18528
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-18524:
Summary: Scala varargs cause exception for new inference
Key: FLINK-18524
URL: https://issues.apache.org/jira/browse/FLINK-18524
Project: Flink
Issue Type
Timo Walther created FLINK-18425:
Summary: GenericArrayData cannot convert object arrays to
primitive arrays
Key: FLINK-18425
URL: https://issues.apache.org/jira/browse/FLINK-18425
Project: Flink
Timo Walther created FLINK-18417:
Summary: Support List as a conversion class for ARRAY
Key: FLINK-18417
URL: https://issues.apache.org/jira/browse/FLINK-18417
Project: Flink
Issue Type: Sub
xecution modes.
Or Flink core just defines the syntax, provides parser and supports a
default execution mode.
The downstream projects can use the APIs and parsed results to decide how
to execute a sql.
Best,
Godfrey
Timo Walther 于2020年6月17日周三 下午6:32写道:
Hi Fabian,
thanks for the proposal. I agree
Hi Leonard,
thanks for the summary.
After reading all of the previous arguments and working on FLIP-95. I
would also lean towards the conclusion of not adding the TEMPORAL keyword.
After FLIP-95, what we considered as a CREATE TEMPORAL TABLE can be
represented as a CREATE TABLE with PRIMARY
Timo Walther created FLINK-18380:
Summary: Add a table source example
Key: FLINK-18380
URL: https://issues.apache.org/jira/browse/FLINK-18380
Project: Flink
Issue Type: Improvement
Timo Walther created FLINK-18363:
Summary: Add user classloader to context in DeSerializationSchema
Key: FLINK-18363
URL: https://issues.apache.org/jira/browse/FLINK-18363
Project: Flink
Hi Fabian,
thanks for the proposal. I agree that we should have consensus on the
SQL syntax as well and thus finalize the concepts introduced in FLIP-84.
I would favor Jark's proposal. I would like to propose the following syntax:
BEGIN STATEMENT SET;
INSERT INTO ...;
INSERT INTO ...;
Timo Walther created FLINK-18324:
Summary: Translate updated data type and function page into Chinese
Key: FLINK-18324
URL: https://issues.apache.org/jira/browse/FLINK-18324
Project: Flink
Timo Walther created FLINK-18273:
Summary: Test and document remote cluster submission in SQL Client
Key: FLINK-18273
URL: https://issues.apache.org/jira/browse/FLINK-18273
Project: Flink
Timo Walther created FLINK-18248:
Summary: Update data type documentation for 1.11
Key: FLINK-18248
URL: https://issues.apache.org/jira/browse/FLINK-18248
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-18090:
Summary: Support RowKind in Row.toString
Key: FLINK-18090
URL: https://issues.apache.org/jira/browse/FLINK-18090
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-18066:
Summary: Add documentation for how to develop a new table
source/sink
Key: FLINK-18066
URL: https://issues.apache.org/jira/browse/FLINK-18066
Project: Flink
Timo Walther created FLINK-18065:
Summary: Add documentation for new scalar/table functions
Key: FLINK-18065
URL: https://issues.apache.org/jira/browse/FLINK-18065
Project: Flink
Issue Type
Timo Walther created FLINK-18005:
Summary: Implement type inference for CAST
Key: FLINK-18005
URL: https://issues.apache.org/jira/browse/FLINK-18005
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-18001:
Summary: Add a new test base for evaluating expressions
Key: FLINK-18001
URL: https://issues.apache.org/jira/browse/FLINK-18001
Project: Flink
Issue Type
Timo Walther created FLINK-17936:
Summary: Implement type inference for AS
Key: FLINK-17936
URL: https://issues.apache.org/jira/browse/FLINK-17936
Project: Flink
Issue Type: Sub-task
LL) | (NULL)
| f4.nested.rowtime - INTERVAL '3' SECOND |
++---+---+---+-+---+
Thanks,
Fabian
Am Mi., 6. Mai 2020 um 17:51 Uhr schrieb godfrey he <
godfre...@gmail.com
:
Hi @fhue...@gmail.com @Timo Walther @Dawid
Wysakowicz
What do you think we li
Timo Walther created FLINK-17882:
Summary: Don't allow self referencing structured type
Key: FLINK-17882
URL: https://issues.apache.org/jira/browse/FLINK-17882
Project: Flink
Issue Type: Sub
Timo Walther created FLINK-17880:
Summary: Use new type inference for SQL table and scalar functions
Key: FLINK-17880
URL: https://issues.apache.org/jira/browse/FLINK-17880
Project: Flink
Timo Walther created FLINK-17879:
Summary: Update Python row serializer
Key: FLINK-17879
URL: https://issues.apache.org/jira/browse/FLINK-17879
Project: Flink
Issue Type: Improvement
Timo Walther created FLINK-17793:
Summary: Replace TableSchema with dedicated CatalogSchema
Key: FLINK-17793
URL: https://issues.apache.org/jira/browse/FLINK-17793
Project: Flink
Issue Type
Timo Walther created FLINK-17716:
Summary: Remove deprecated RowData.get and similar methods
Key: FLINK-17716
URL: https://issues.apache.org/jira/browse/FLINK-17716
Project: Flink
Issue Type
Timo Walther created FLINK-17698:
Summary: Support for comparing instance of all types
Key: FLINK-17698
URL: https://issues.apache.org/jira/browse/FLINK-17698
Project: Flink
Issue Type: Bug
Timo Walther created FLINK-17668:
Summary: Fix shortcomings in new data structures
Key: FLINK-17668
URL: https://issues.apache.org/jira/browse/FLINK-17668
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-17660:
Summary: DataTypeExtractor does not check for default constructor
Key: FLINK-17660
URL: https://issues.apache.org/jira/browse/FLINK-17660
Project: Flink
:
format.kind: json
format.fail-on-missing-field: true
If fail-on-missing-field is specific for json, then one could go with
format: json
json.fail-on-missing-field: true
or
format.kind: json
format.json.fail-on-missing-field: true
Cheers,
Till
On Fri, May 1, 2020 at 11:55 AM Timo W
Timo Walther created FLINK-17541:
Summary: Support inline structured types
Key: FLINK-17541
URL: https://issues.apache.org/jira/browse/FLINK-17541
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-17532:
Summary: Update tests to use BatchTestBase#checkTableResult
Key: FLINK-17532
URL: https://issues.apache.org/jira/browse/FLINK-17532
Project: Flink
Issue
Timo Walther created FLINK-17521:
Summary: Remove `com.ibm.icu` dependency from table-common
Key: FLINK-17521
URL: https://issues.apache.org/jira/browse/FLINK-17521
Project: Flink
Issue Type
Timo Walther created FLINK-17489:
Summary: Support any kind of array in
StringUtils.arrayAwareToString()
Key: FLINK-17489
URL: https://issues.apache.org/jira/browse/FLINK-17489
Project: Flink
;
> 'format.kind' = 'json',
> 'format.fail-on-missing-field' = 'false'
Yes. That's pretty much it.
This is reasonable important to nail down as with such violations I
believe we could not actually switch to a standard YAML parser.
On 29/04/2020 16:05, Timo Walther wrote:
Hi everyone,
dis
Timo Walther created FLINK-17484:
Summary: Enable type coercion
Key: FLINK-17484
URL: https://issues.apache.org/jira/browse/FLINK-17484
Project: Flink
Issue Type: Sub-task
Hi Leonard,
this sounds like a nice refactoring for consistency. +1 from my side.
However, I'm not sure how much backwards compatibility is required.
Maybe others can comment on this.
Thanks,
Timo
On 30.04.20 14:09, Leonard Xu wrote:
Hi, dear community
Recently, I’m thinking to refactor
Hi Xuannan,
sorry, for not entering the discussion earlier. Could you please update
the FLIP to how it would like after FLIP-84? I think your proposal makes
sense to me and aligns well with the other efforts from an API
perspective. However, here are some thought from my side:
It would be
Hi everyone,
discussions around ConfigOption seem to be very popular recently. So I
would also like to get some opinions on a different topic.
How do we represent hierarchies in ConfigOption? In FLIP-122, we agreed
on the following DDL syntax:
CREATE TABLE fs_table (
...
) WITH (
Timo Walther created FLINK-17373:
Summary: Support the NULL type for function calls
Key: FLINK-17373
URL: https://issues.apache.org/jira/browse/FLINK-17373
Project: Flink
Issue Type: Sub
Timo Walther created FLINK-17370:
Summary: Ensure deterministic type inference extraction
Key: FLINK-17370
URL: https://issues.apache.org/jira/browse/FLINK-17370
Project: Flink
Issue Type
From the SQL side, I'm sure that FLIP-95 and FLIP-105 could benefit
from extending the feature freeze.
Thanks,
Timo
On 23.04.20 16:11, Aljoscha Krettek wrote:
+1
Aljoscha
On 23.04.20 15:23, Till Rohrmann wrote:
+1 for extending the feature freeze until May 15th.
Cheers,
Till
On Thu, Apr
+1 (binding)
Thanks,
Timo
On 16.04.20 11:32, Benchao Li wrote:
+1 (non-binding)
Aljoscha Krettek 于2020年4月16日周四 下午5:10写道:
+1 (binding)
Aljoscha
Timo Walther created FLINK-17190:
Summary: SQL client does not support views that reference a table
from DDL
Key: FLINK-17190
URL: https://issues.apache.org/jira/browse/FLINK-17190
Project: Flink
Timo Walther created FLINK-17158:
Summary: Watermark strategy property cannot be expressed in YAML
Key: FLINK-17158
URL: https://issues.apache.org/jira/browse/FLINK-17158
Project: Flink
+1 (binding)
Thanks for the healthy discussion. I think this feature can be useful
during the development of a pipeline.
Regards,
Timo
On 10.04.20 03:34, Danny Chan wrote:
Hi all,
I would like to start the vote for FLIP-113 [1], which is discussed and
reached a consensus in the discussion
Hi,
welcome to our community! You don't need to be a contributor in JIRA to
contribute to Apache Flink. Just ping a committer if your want to work
on some JIRA ticket and someone will assign the ticket to you.
Regards,
Timo
On 09.04.20 11:03, zhangzhanhua wrote:
Hi Guys,
I want to
+1 (binding)
Thanks for your efforts.
Regards,
Timo
On 09.04.20 14:46, Zhenghua Gao wrote:
Hi all,
I'd like to start the vote for FLIP-71[1] which adds E2E view support in
Flink SQL.
This FLIP is discussed in the thread[2].
The vote will be open for at least 72 hours. Unless there is an
FLIP.
And may be we should open a separate JIRA ticket to track supporting it in
table DDL?
*Best Regards,*
*Zhenghua Gao*
On Wed, Apr 8, 2020 at 7:48 PM Timo Walther wrote:
Hi Zhenghua,
FLINK-10232 is quite old and a lot of stuff was discussed and agreed on
since then. I don't like to pos
wrote:
Hi Timo,
Shall we postpone the support of 'TEMPORARY' keyword since it's not
mentioned in FLINK-10232?
<https://issues.apache.org/jira/browse/FLINK-10232>
*Best Regards,*
*Zhenghua Gao*
On Wed, Apr 8, 2020 at 3:30 PM Timo Walther wrote:
Hi Zhenghua,
VIEWS should also suppor
Hi Zhenghua,
VIEWS should also support the TEMPORARY keyword according to FLIP-64.
Otherwise the FLIP looks good to me.
Regards,
Timo
On 08.04.20 07:31, Zhenghua Gao wrote:
@Danny Chan you‘re right. I have updated the doc.
*Best Regards,*
*Zhenghua Gao*
On Wed, Apr 8, 2020 at 1:20 PM
onable compromise, disabling this feature by default
is a
way to protect
the vulnerability, and we can simplify the design quite a lot. We can
gather some users'
feedback to see whether further protections are necessary in the future.
Best,
Kurt
On Mon, Apr 6, 2020 at 11:49 PM Timo Walther wrot
Timo Walther created FLINK-17030:
Summary: Add primary key syntax to DDL
Key: FLINK-17030
URL: https://issues.apache.org/jira/browse/FLINK-17030
Project: Flink
Issue Type: Sub-task
the duplicate keys
with #supportedKeys .(Not very strong reason, but i think as a
connector developer, it makes sense)
Best,
Danny Chan
在 2020年4月3日 +0800 PM11:28,Timo Walther ,写道:
Hi everyone,
@Aljoscha: I disagree with your approach because a `Catalog` can return
a custom factory that is not using
Thanks, for the update.
+1 (binding) for this FLIP
Regards,
Timo
On 06.04.20 16:47, godfrey he wrote:
Hi Timo,
Sorry for late reply, and thanks for your correction. I have fixed the typo
and updated the document.
Best,
Godfrey
Timo Walther 于2020年4月6日周一 下午6:05写道:
Hi Godfrey,
did you
. Let's wait for 24h.
Best,
Godfrey
Timo Walther 于2020年3月31日周二 下午5:26写道:
-1
The current discussion has not completed. The last comments were sent
less than 24h ago.
Let's wait a bit longer to collect feedback from all stakeholders.
Thanks,
Timo
On 31.03.20 08:31, godfrey he wrote:
Hi everyone
Hi Dawid,
thanks for this FLIP. This solves a lot of issues with the current
design for both the Flink contributors and users. +1 for this.
Some minor suggestions from my side:
- How about finding something shorter for `InitializationContext`? Maybe
just `OpenContext`?
- While introducing
Timo Walther created FLINK-17000:
Summary: Type information in sources should cover all data
structures
Key: FLINK-17000
URL: https://issues.apache.org/jira/browse/FLINK-17000
Project: Flink
Timo Walther created FLINK-16999:
Summary: Data structure should cover all conversions declared in
logical types
Key: FLINK-16999
URL: https://issues.apache.org/jira/browse/FLINK-16999
Project: Flink
Timo Walther created FLINK-16998:
Summary: Add a changeflag to Row type
Key: FLINK-16998
URL: https://issues.apache.org/jira/browse/FLINK-16998
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-16997:
Summary: Add new factory interfaces and utilities
Key: FLINK-16997
URL: https://issues.apache.org/jira/browse/FLINK-16997
Project: Flink
Issue Type: Sub
Timo Walther created FLINK-16996:
Summary: Refactor planner and connectors to use new data structures
Key: FLINK-16996
URL: https://issues.apache.org/jira/browse/FLINK-16996
Project: Flink
Timo Walther created FLINK-16995:
Summary: Add new data structure interfaces in table-common
Key: FLINK-16995
URL: https://issues.apache.org/jira/browse/FLINK-16995
Project: Flink
Issue Type
Timo Walther created FLINK-16994:
Summary: Support SupportsWatermarkPushDown in planner
Key: FLINK-16994
URL: https://issues.apache.org/jira/browse/FLINK-16994
Project: Flink
Issue Type: Sub
Timo Walther created FLINK-16993:
Summary: Support SupportsComputedColumnPushDown in planner
Key: FLINK-16993
URL: https://issues.apache.org/jira/browse/FLINK-16993
Project: Flink
Issue Type
Timo Walther created FLINK-16992:
Summary: Add ability interfaces for table source/sink
Key: FLINK-16992
URL: https://issues.apache.org/jira/browse/FLINK-16992
Project: Flink
Issue Type: Sub
Timo Walther created FLINK-16991:
Summary: Support DynamicTableSink in planner
Key: FLINK-16991
URL: https://issues.apache.org/jira/browse/FLINK-16991
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-16990:
Summary: Support LookupTableSource in planner
Key: FLINK-16990
URL: https://issues.apache.org/jira/browse/FLINK-16990
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-16989:
Summary: Support ScanTableSource in planner
Key: FLINK-16989
URL: https://issues.apache.org/jira/browse/FLINK-16989
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-16988:
Summary: Add core table source/sink interfaces
Key: FLINK-16988
URL: https://issues.apache.org/jira/browse/FLINK-16988
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-16987:
Summary: Add new table source and sink interfaces
Key: FLINK-16987
URL: https://issues.apache.org/jira/browse/FLINK-16987
Project: Flink
Issue Type: New
501 - 600 of 1353 matches
Mail list logo