[jira] [Created] (FLINK-35437) BlockStatementGrouper uses lots of memory

2024-05-23 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-35437: Summary: BlockStatementGrouper uses lots of memory Key: FLINK-35437 URL: https://issues.apache.org/jira/browse/FLINK-35437 Project: Flink Issue Type

[jira] [Created] (FLINK-35216) Support for RETURNING clause of JSON_QUERY

2024-04-23 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-35216: Summary: Support for RETURNING clause of JSON_QUERY Key: FLINK-35216 URL: https://issues.apache.org/jira/browse/FLINK-35216 Project: Flink Issue

[jira] [Created] (FLINK-35155) Introduce TableRuntimeException

2024-04-18 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-35155: Summary: Introduce TableRuntimeException Key: FLINK-35155 URL: https://issues.apache.org/jira/browse/FLINK-35155 Project: Flink Issue Type: New

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-04-10 Thread Dawid Wysakowicz
Hi all, I thought I'd cast my vote as well to give extra data: 1. Materialized Table 2. Materialized View (generally speaking I am not too concerned with using a View here, but since there are concerns around updating a view I put it second) I think what is suggested in this FLIP is

[jira] [Created] (FLINK-35021) AggregateQueryOperations produces wrong asSerializableString representation

2024-04-05 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-35021: Summary: AggregateQueryOperations produces wrong asSerializableString representation Key: FLINK-35021 URL: https://issues.apache.org/jira/browse/FLINK-35021

[jira] [Created] (FLINK-34938) Incorrect behaviour for comparison functions

2024-03-26 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-34938: Summary: Incorrect behaviour for comparison functions Key: FLINK-34938 URL: https://issues.apache.org/jira/browse/FLINK-34938 Project: Flink Issue

[jira] [Created] (FLINK-34910) Can not plan window join without projections

2024-03-21 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-34910: Summary: Can not plan window join without projections Key: FLINK-34910 URL: https://issues.apache.org/jira/browse/FLINK-34910 Project: Flink Issue

[jira] [Created] (FLINK-34745) Parsing temporal table join throws cryptic exceptions

2024-03-19 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-34745: Summary: Parsing temporal table join throws cryptic exceptions Key: FLINK-34745 URL: https://issues.apache.org/jira/browse/FLINK-34745 Project: Flink

[jira] [Created] (FLINK-34507) JSON functions have wrong operand checker

2024-02-23 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-34507: Summary: JSON functions have wrong operand checker Key: FLINK-34507 URL: https://issues.apache.org/jira/browse/FLINK-34507 Project: Flink Issue Type

[jira] [Created] (FLINK-34399) Release Testing: Verify FLINK-33644 Make QueryOperations SQL serializable

2024-02-06 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-34399: Summary: Release Testing: Verify FLINK-33644 Make QueryOperations SQL serializable Key: FLINK-34399 URL: https://issues.apache.org/jira/browse/FLINK-34399

[jira] [Created] (FLINK-34153) Set ALWAYS ChainingStrategy in TemporalSort

2024-01-18 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-34153: Summary: Set ALWAYS ChainingStrategy in TemporalSort Key: FLINK-34153 URL: https://issues.apache.org/jira/browse/FLINK-34153 Project: Flink Issue

Re: Re: [VOTE] Accept Flink CDC into Apache Flink

2024-01-10 Thread Dawid Wysakowicz
+1 (binding) Best, Dawid On Wed, 10 Jan 2024 at 11:54, Piotr Nowojski wrote: > +1 (binding) > > śr., 10 sty 2024 o 11:25 Martijn Visser > napisał(a): > > > +1 (binding) > > > > On Wed, Jan 10, 2024 at 4:43 AM Xingbo Huang wrote: > > > > > > +1 (binding) > > > > > > Best, > > > Xingbo > > > >

Re: [DISCUSS][FLINK-31830] Align the Nullability Handling of ROW between SQL and TableAPI

2024-01-09 Thread Dawid Wysakowicz
Hey all, First of all, sorry I have not read the entire thread. I just wanted to make sure you take this one case into consideration. As far as I know, we map java classes to SQL ROWs? E.g. it is possible to have a POJO as a parameter to a UDF. *class MyUDF {* * eval(MyPojo a)* *}* *class

Re: [VOTE] Release flink-shaded 18.0, release candidate #1

2024-01-08 Thread Dawid Wysakowicz
+1 (binding) - Validated hashes - Verified signature - Verified that no binaries exist in the source archive Best, Dawid On Tue, 28 Nov 2023 at 23:10, Sergey Nuyanzin wrote: > Hi everyone, > Please review and vote on the release candidate #1 for the version 18.0, as > follows: > [ ] +1,

[jira] [Created] (FLINK-33823) Serialize PlannerQueryOperation into SQL

2023-12-14 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33823: Summary: Serialize PlannerQueryOperation into SQL Key: FLINK-33823 URL: https://issues.apache.org/jira/browse/FLINK-33823 Project: Flink Issue Type

[jira] [Created] (FLINK-33754) Serialize QueryOperations into SQL

2023-12-05 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33754: Summary: Serialize QueryOperations into SQL Key: FLINK-33754 URL: https://issues.apache.org/jira/browse/FLINK-33754 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-33675) Serialize ValueLiteralExpressions into SQL

2023-11-28 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33675: Summary: Serialize ValueLiteralExpressions into SQL Key: FLINK-33675 URL: https://issues.apache.org/jira/browse/FLINK-33675 Project: Flink Issue

[jira] [Created] (FLINK-33663) Serialize CallExpressions into SQL

2023-11-27 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33663: Summary: Serialize CallExpressions into SQL Key: FLINK-33663 URL: https://issues.apache.org/jira/browse/FLINK-33663 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-33644) FLIP-393: Make QueryOperations SQL serializable

2023-11-24 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33644: Summary: FLIP-393: Make QueryOperations SQL serializable Key: FLINK-33644 URL: https://issues.apache.org/jira/browse/FLINK-33644 Project: Flink

[RESULT][VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-24 Thread Dawid Wysakowicz
(binding) Dawid Wysakowicz (binding) Ferenc Csaky (non-binding) Xiangyu Feng (non-binding) Jim Hughes (non-binding) Thanks to all involved. [1] https://cwiki.apache.org/confluence/x/vQ2ZE [2] https://lists.apache.org/thread/5txkmqx9wfj0lzg02vnrw99cj63b5zvj Best, Dawid

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-24 Thread Dawid Wysakowicz
> > > > > > > > > > > > > > Lookgin forward to this! > > > > > > > > > > > > > > Best, > > > > > > > Ferenc > > > > > > > > > > > > >

[VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread Dawid Wysakowicz
Hi everyone, Thank you to everyone for the feedback on FLIP-393: Make QueryOperations SQL serializable[1] which has been discussed in this thread [2]. I would like to start a vote for it. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1]

[jira] [Created] (FLINK-33599) Run restore tests with RocksDB state backend

2023-11-20 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33599: Summary: Run restore tests with RocksDB state backend Key: FLINK-33599 URL: https://issues.apache.org/jira/browse/FLINK-33599 Project: Flink Issue

Re: [DISCUSS] FLIP-393: Make QueryOperations SQL serializable

2023-11-20 Thread Dawid Wysakowicz
proposal. > > Regards, > Xiangyu > > Benchao Li 于2023年11月17日周五 19:07写道: > > > That sounds good to me, I'm looking forward to it! > > > > After this FLIP is done, FLINK-25015 can utilize this ability to set > > job name for queries. > > >

[jira] [Created] (FLINK-33597) Can not use a nested column for a join condition

2023-11-20 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33597: Summary: Can not use a nested column for a join condition Key: FLINK-33597 URL: https://issues.apache.org/jira/browse/FLINK-33597 Project: Flink

Re: [DISCUSS] FLIP-393: Make QueryOperations SQL serializable

2023-11-16 Thread Dawid Wysakowicz
some exceptions such as > PlannerQueryOperation. > > [1] > https://github.com/apache/calcite/blob/153796f8994831ad015af4b9036aa01ebf78/core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java#L141 > > Dawid Wysakowicz 于2023年11月16日周四 16:24写道: > > > > I think the FLIP co

Re: [DISCUSS] FLIP-393: Make QueryOperations SQL serializable

2023-11-16 Thread Dawid Wysakowicz
preliminary state, you just want to gather ideas > about whether to use SQL or something else as the serialization format > for QueryOperation? > > Dawid Wysakowicz 于2023年11月15日周三 19:34写道: > > > > Hi, > > I would like to propose a follow-up improvement to some o

[jira] [Created] (FLINK-33563) Implement type inference for Agg functions

2023-11-15 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33563: Summary: Implement type inference for Agg functions Key: FLINK-33563 URL: https://issues.apache.org/jira/browse/FLINK-33563 Project: Flink Issue

[DISCUSS] FLIP-393: Make QueryOperations SQL serializable

2023-11-15 Thread Dawid Wysakowicz
Hi, I would like to propose a follow-up improvement to some of the work that has been done over the years to the Table API. I posted the proposed changes in the FLIP-393. I'd like to get to know what others think of choosing SQL as the serialization format for QueryOperations. Regards, Dawid

Re: [VOTE] FLIP-376: Add DISTRIBUTED BY clause

2023-11-06 Thread Dawid Wysakowicz
+1 Best, Dawid On Mon, 6 Nov 2023 at 12:38, Timo Walther wrote: > Hi everyone, > > I'd like to start a vote on FLIP-376: Add DISTRIBUTED BY clause[1] which > has been discussed in this thread [2]. > > The vote will be open for at least 72 hours unless there is an objection > or not enough

[jira] [Created] (FLINK-33439) Implement type inference for IN function

2023-11-02 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33439: Summary: Implement type inference for IN function Key: FLINK-33439 URL: https://issues.apache.org/jira/browse/FLINK-33439 Project: Flink Issue Type

[jira] [Created] (FLINK-33431) Create restore tests for ExecNodes

2023-11-02 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33431: Summary: Create restore tests for ExecNodes Key: FLINK-33431 URL: https://issues.apache.org/jira/browse/FLINK-33431 Project: Flink Issue Type

[jira] [Created] (FLINK-33419) Port PROCTIME/ROWTIME functions to the new inference stack

2023-10-31 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33419: Summary: Port PROCTIME/ROWTIME functions to the new inference stack Key: FLINK-33419 URL: https://issues.apache.org/jira/browse/FLINK-33419 Project: Flink

[jira] [Created] (FLINK-33412) Implement type inference for reinterpret_cast function

2023-10-31 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33412: Summary: Implement type inference for reinterpret_cast function Key: FLINK-33412 URL: https://issues.apache.org/jira/browse/FLINK-33412 Project: Flink

[jira] [Created] (FLINK-33411) Implement type inference for window properties functions

2023-10-31 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33411: Summary: Implement type inference for window properties functions Key: FLINK-33411 URL: https://issues.apache.org/jira/browse/FLINK-33411 Project: Flink

[jira] [Created] (FLINK-33410) Implement type inference for Over offsets functions

2023-10-31 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33410: Summary: Implement type inference for Over offsets functions Key: FLINK-33410 URL: https://issues.apache.org/jira/browse/FLINK-33410 Project: Flink

[jira] [Created] (FLINK-33407) Port time functions to the new type inference stack

2023-10-31 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33407: Summary: Port time functions to the new type inference stack Key: FLINK-33407 URL: https://issues.apache.org/jira/browse/FLINK-33407 Project: Flink

[jira] [Created] (FLINK-33375) Add a RestoreTestBase

2023-10-26 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33375: Summary: Add a RestoreTestBase Key: FLINK-33375 URL: https://issues.apache.org/jira/browse/FLINK-33375 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-33372) Cryptic exception for a sub query in a CompiledPlan

2023-10-26 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33372: Summary: Cryptic exception for a sub query in a CompiledPlan Key: FLINK-33372 URL: https://issues.apache.org/jira/browse/FLINK-33372 Project: Flink

[jira] [Created] (FLINK-33371) Make TestValues sinks return results as Rows

2023-10-26 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33371: Summary: Make TestValues sinks return results as Rows Key: FLINK-33371 URL: https://issues.apache.org/jira/browse/FLINK-33371 Project: Flink Issue

[jira] [Created] (FLINK-33327) Window TVF column expansion does not work with an INSERT INTO

2023-10-20 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33327: Summary: Window TVF column expansion does not work with an INSERT INTO Key: FLINK-33327 URL: https://issues.apache.org/jira/browse/FLINK-33327 Project: Flink

[jira] [Created] (FLINK-33255) Validate argument count during type inference

2023-10-12 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33255: Summary: Validate argument count during type inference Key: FLINK-33255 URL: https://issues.apache.org/jira/browse/FLINK-33255 Project: Flink Issue

[jira] [Created] (FLINK-33223) MATCH_RECOGNIZE AFTER MATCH clause can not be deserialised from a compiled plan

2023-10-09 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33223: Summary: MATCH_RECOGNIZE AFTER MATCH clause can not be deserialised from a compiled plan Key: FLINK-33223 URL: https://issues.apache.org/jira/browse/FLINK-33223

Re: [DISCUSS] [FLINK-32873] Add a config to allow disabling Query hints

2023-10-05 Thread Dawid Wysakowicz
wrote: Hi Dawid, Thanks for bringing this. I would agree with enum approach ignored option would allow to follow Oracle's behavior as well table.optimizer.query-options = ENABLED/DISABLED/IGNORED nit: Can we have "hint" in config option name e.g. table.optimizer.query-options-hints ?

Re: [DISCUSS] [FLINK-32873] Add a config to allow disabling Query hints

2023-10-03 Thread Dawid Wysakowicz
Hey all, My understanding was that from the first message we were discussing throwing an exception. Oracle was only shown as an example of a system that have a flag for hints behaviour. Let's get back to the discussion and agree on the behavior. My suggestion is to introduce an enum instead

[jira] [Created] (FLINK-33179) Improve reporting serialisation issues

2023-10-03 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33179: Summary: Improve reporting serialisation issues Key: FLINK-33179 URL: https://issues.apache.org/jira/browse/FLINK-33179 Project: Flink Issue Type

[jira] [Created] (FLINK-33158) Cryptic exception when there is a StreamExecSort in JsonPlan

2023-09-26 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33158: Summary: Cryptic exception when there is a StreamExecSort in JsonPlan Key: FLINK-33158 URL: https://issues.apache.org/jira/browse/FLINK-33158 Project: Flink

[jira] [Created] (FLINK-33093) SHOW FUNCTIONS throw exception with unset catalog

2023-09-15 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33093: Summary: SHOW FUNCTIONS throw exception with unset catalog Key: FLINK-33093 URL: https://issues.apache.org/jira/browse/FLINK-33093 Project: Flink

[jira] [Created] (FLINK-33083) SupportsReadingMetadata is not applied when loading a CompiledPlan

2023-09-13 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33083: Summary: SupportsReadingMetadata is not applied when loading a CompiledPlan Key: FLINK-33083 URL: https://issues.apache.org/jira/browse/FLINK-33083 Project

[jira] [Created] (FLINK-32682) Introduce option for choosing time function evaluation methods

2023-07-26 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-32682: Summary: Introduce option for choosing time function evaluation methods Key: FLINK-32682 URL: https://issues.apache.org/jira/browse/FLINK-32682 Project

[jira] [Created] (FLINK-32584) Make it possible to unset default catalog and/or database

2023-07-12 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-32584: Summary: Make it possible to unset default catalog and/or database Key: FLINK-32584 URL: https://issues.apache.org/jira/browse/FLINK-32584 Project: Flink

Re: Confluent Avro and Debezium formats - default schema name can be incompatible with registered schema name

2023-01-27 Thread Dawid Wysakowicz
Hi Fruzsina, I think this is a valid issue we should try to solve. A different approach I am thinking about is that we could actually add an option to provide an entire avro schema to use. Something like: `avro-confluent.schema` which we would validate it maps properly to the schema of the

Re: [VOTE] Release 1.16.1, release candidate #1

2023-01-26 Thread Dawid Wysakowicz
+1 (binding) * verified checksums & signatures * checked differences of pom.xml and NOTICE files with 1.16.0 release. looks good * checked source release contains no binaries * built from sources * run StateMachineExample on a local cluster * checked the web PR Best, Dawid On

Re: [VOTE] FLIP-290: Operator state compression (FLINK-30113)

2023-01-20 Thread Dawid Wysakowicz
+1 (binding) Best, Dawid On 20/01/2023 11:24, Etienne Chauchot wrote: Hi all, after the discussion in [1], I would like to open a voting thread for FLIP-290 [2] which discusses operator state compression. The vote will be open until January 25th (72h + weekend), unless there is an

Re: [DISCUSS] FLIP-290: Operator state compression (FLINK-30113)

2023-01-20 Thread Dawid Wysakowicz
compression is a good start if we really want to introduce more compression formats in the future. Best Yun Tang *From:* Dawid Wysakowicz *Sent:* Friday, January 20, 2023 16:41 *To:* dev@flink.apache.org; Yun Tang *Subject

Re: [DISCUSS] FLIP-290: Operator state compression (FLINK-30113)

2023-01-20 Thread Dawid Wysakowicz
: /UncompressedStreamCompressionDecorator/ and /SnappyStreamCompressionDecorator/. But I agree, I'd stick to /SnappyStreamCompressionDecorator /which is in use for keyed state compression. I'll add a comment on the FLIP saying that. Best Etienne Le 18/01/2023 à 16:36, Dawid Wysakowicz a écrit

Re: [DISCUSS] FLIP-290: Operator state compression (FLINK-30113)

2023-01-18 Thread Dawid Wysakowicz
question about this flip - Can we select the compression format, for example? such as snappy or lz0 - If compression format is supported, I think the metadata will probably have to include which compression format and I agree idea with Dawid Dawid Wysakowicz 于2023年1月18日周三 20:55写道

Re: [DISCUSS] FLIP-290: Operator state compression (FLINK-30113)

2023-01-18 Thread Dawid Wysakowicz
It makes sense from my side. Could you, just for completeness, extend it with the info what will be the compression unit? If understand it correctly you envision each state to be compressed separately. Best, Dawid On 17/01/2023 15:47, Etienne Chauchot wrote: Hi everyone, I just published

Re: [DISCUSS] FLIP-286: Fix the API stability/scope annotation inconsistency in AbstractStreamOperator

2023-01-16 Thread Dawid Wysakowicz
gt; framework primitives we want to expose to the users. So instead we just > expose a bunch of internal things and hope users only use the "stable" part > of them. > > Thanks, > > Jiangjie (Becket) Qin > > > On Fri, Jan 13, 20

Re: [DISCUSS] FLIP-286: Fix the API stability/scope annotation inconsistency in AbstractStreamOperator

2023-01-13 Thread Dawid Wysakowicz
Hi Becket, May I ask what is the motivation for the change? I'm really skeptical about making any of those classes `Public` or `PublicEvolving`. As far as I am concerned there is no agreement in the community if StreamOperator is part of the `Public(Evolving)` API. At least I think it should

Re: [VOTE] Release flink-connector-rabbitmq, release candidate #1

2022-12-20 Thread Dawid Wysakowicz
+1 (binding) * Validated hashes * Verified signature * Verified that no binaries exist in the source archive * Built the source with Maven * Verified licenses * Verified web PR Best, Dawid On 20/12/2022 13:04, Martijn Visser wrote: +1 (binding) - Validated hashes - Verified signature

Re: [VOTE] Release flink-connector-jdbc, release candidate #2

2022-12-19 Thread Dawid Wysakowicz
+1 (binding) * verified signatures and checksums * built from sources * the PR looks good * the artifacts to be published look fine * checked files diff to the 1.16.0 release tag Best, Dawid On 19/12/2022 12:17, Sergey Nuyanzin wrote: +1 (non-binding) - Validated hashes and signature -

Re: [VOTE] Release flink-connector-pulsar, release candidate #5

2022-12-19 Thread Dawid Wysakowicz
+1 (binding) * verified signatures and checksums * built from sources * checked notice files * the PR looks good * the artifacts to be published look fine Best, Dawid On 15/12/2022 16:41, Martijn Visser wrote: Hi everyone, Please review and vote on the release candidate #5 for the

[jira] [Created] (FLINK-30427) Pulsar SQL connector lists not bundled dependencies

2022-12-15 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-30427: Summary: Pulsar SQL connector lists not bundled dependencies Key: FLINK-30427 URL: https://issues.apache.org/jira/browse/FLINK-30427 Project: Flink

Re: [DISCUSS] FLIP-269: Properly Handling the Processing Timers on Job Termination

2022-12-08 Thread Dawid Wysakowicz
on. Best, Yun Gao --Original Mail -- *Sender:*Dawid Wysakowicz *Send Date:*Fri Dec 2 21:21:25 2022 *Recipients:*Dev *Subject:*Re: [DISCUSS] FLIP-269: Properly Handling the Processing Timers on Job Termination Ad. 1

Re: [DISCUSS] FLIP-269: Properly Handling the Processing Timers on Job Termination

2022-12-02 Thread Dawid Wysakowicz
be without this functionality and we can add it later (given that we designed a good place to add it in the future, like the `Context` proposed by Dawid). But I'm also fine adding it now if others are insisting. Best, Piotrek śr., 30 lis 2022 o 09:18 Dawid Wysakowicz napisał(a): WindowOperator is not implemente

Re: [DISCUSS] FLIP-269: Properly Handling the Processing Timers on Job Termination

2022-11-30 Thread Dawid Wysakowicz
rators. Just that each built-in operator can have a hard coded way (without a way for users to change it) to handle those timers. Windowed operators would trigger the lingering timers (flush outputs), AsyncWaitOperator could just ignore them. The same way users could register EOF timer handlers in

Re: [VOTE] Release flink-connector-cassandra, release candidate #1

2022-11-29 Thread Dawid Wysakowicz
+1 (binding) - Downloaded artifacts - Checked hash and signature - No binaries in source archive found - Verified NOTICE files - Built from source code - Verified that no SNAPSHOT versions exist, all versions point to 3.0.0 in POM files - Tag is OK - Reviewed the Web PR Few notes: - the link

Re: [DISCUSS] FLIP-269: Properly Handling the Processing Timers on Job Termination

2022-11-28 Thread Dawid Wysakowicz
EOF timer handlers in the ProcessFunction as Dawid Wysakowicz proposed, we (as flink developers) could use the same mechanism to implement any behaviour we want for the built-in operators. There should be no need to add any separate mechanism. Best, Piotrek śr., 23 lis 2022 o 08:21 Divye Kapo

Re: Stack Overflow Question re compressing broadcast state

2022-11-22 Thread Dawid Wysakowicz
. On Nov 21, 2022, at 1:36 AM, Dawid Wysakowicz wrote: And yes, I read "Compression works on the granularity of key-groups in keyed state” as meaning “When compressing keyed state, it’s done per key-group” and not “Compression only works on keyed state” :) Totally agree. Docs

Re: [DISCUSS] FLIP-269: Properly Handling the Processing Timers on Job Termination

2022-11-21 Thread Dawid Wysakowicz
I see the problem differently. I really don't think this should be looked at from the perspective of timers, but rather from the semantics of endOfInput. If a proc time timer is set for 10s, the 10s are "real-time" and the delay is important. One particularly important example is

Re: Stack Overflow Question re compressing broadcast state

2022-11-21 Thread Dawid Wysakowicz
7, 2022, at 12:06 AM, Dawid Wysakowicz wrote: Cross posting answer from SO: BroadcastState is an operator state not a KeyedState. The referenced docs refer to a KeyedState: Compression works on the granularity of key-groups in keyed state, Probably docs could be more explicit about this behav

Re: Stack Overflow Question re compressing broadcast state

2022-11-17 Thread Dawid Wysakowicz
Hoping someone (like Dawid Wysakowicz) can chime in here, thanks! — Ken -- Ken Krugler http://www.scaleunlimited.com Custom big data solutions Flink, Pinot, Solr, Elasticsearch OpenPGP_0x31D2DD10BFC15A2D.asc Description: OpenPGP public key OpenPGP_signature Descr

Re: [DISCUSS] FLIP-269: Properly Handling the Processing Timers on Job Termination

2022-11-15 Thread Dawid Wysakowicz
Hey all, Sorry for being rather late to the party. I'd like to chime in a few thoughts. I agree there are at least two aspects to the problem: 1) lower level handling (StreamOperator) 2) higher level APIs (WindowOperator, CEP, ProcessFunction, ...) First is how do we want to deal with it

Re: [VOTE] Drop TypeSerializerConfigSnapshot and savepoint support from Flink versions < 1.8.0

2022-10-31 Thread Dawid Wysakowicz
+1 On 28/10/2022 16:57, Piotr Nowojski wrote: Hi, As discussed on the dev mailing list [0] I would like to start a vote to drop support of older savepoint formats (for Flink versions older than 1.8). You can find the original explanation from the aforementioned dev mailing list thread at the

Re: [VOTE] FLIP-263: Improve resolving schema compatibility

2022-10-28 Thread Dawid Wysakowicz
+1, Best, Dawid On 28/10/2022 08:08, godfrey he wrote: +1 (binding) Thanks for driving this! Best, Godfrey Yun Gao 于2022年10月28日周五 13:50写道: +1 (binding) Thanks Hangxiang for driving the FLIP. Best, Yun Gao --Original Mail -- Sender:Zakelly Lan Send

Re: [DISCUSS] FLIP-263: Improve resolving schema compatibility

2022-10-28 Thread Dawid Wysakowicz
patibility resolveSchemaCompatibility( TypeSerializerSnapshot oldSnapshot); } ` Best, Godfrey Dawid Wysakowicz 于2022年10月20日周四 16:59写道: That's the final status we will arrive at. IIUC, we cannot just remove the original method but just mark it as deprecated so two methods have to exist toget

Re: [DISCUSS] Drop TypeSerializerConfigSnapshot and savepoint support from Flink versions < 1.8.0

2022-10-20 Thread Dawid Wysakowicz
+1, I think that is a sensible think to do. I don't think this will affect many users as those versions are already quite old. As a side note, there is a different effort around serializers that might introduce another incompatibility (in the API). I wonder if we could squash it together

Re: [DISCUSS] FLIP-263: Improve resolving schema compatibility

2022-10-20 Thread Dawid Wysakowicz
their own external serializers which is a long run. I'd like to make jobs where users could always work without modifying any codes before removing the deprecated method so I provide a default implementation in the proposal. On Mon, Oct 17, 2022 at 10:10 PM Dawid Wysakowicz wrote: Hi Han

Re: [DISCUSS] FLIP-263: Improve resolving schema compatibility

2022-10-17 Thread Dawid Wysakowicz
Hi Han, I think in principle your proposal makes sense and the compatibility check indeed should be done in the opposite direction. However, I have two suggestions: 1. Should we introduce the new method to the TypeSerializerSnapshot instead? E.g.    TypeSerializerSnapshot {       

Re: [VOTE] Release flink-shaded 16.0, release candidate #1

2022-10-06 Thread Dawid Wysakowicz
+1 (binding) - Verified that checksums and GPG files match the released files - Checked that POM files point to the same version - Verified licenses - Verified web PR Best, Dawid On 29/09/2022 09:22, Martijn Visser wrote: Thanks for pushing this release Chesnay. +1 (binding) - Verified

[jira] [Created] (FLINK-29500) InitializeOnMaster uses wrong parallelism with AdaptiveScheduler

2022-10-04 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-29500: Summary: InitializeOnMaster uses wrong parallelism with AdaptiveScheduler Key: FLINK-29500 URL: https://issues.apache.org/jira/browse/FLINK-29500 Project

Re: [DISCUSS] FLIP-260: Expose Finish Method For UserDefinedFunction

2022-09-15 Thread Dawid Wysakowicz
Hey Lincoln, Thanks for opening the discussion. To be honest I am not convinced if emitting from close there is a contract that was envisioned and thus should be maintained. As far as I can see it does affect only the TableFunction, because it has the collect method. None of the other UDFs

Re: [VOTE] Release 1.15.2, release candidate #2

2022-08-22 Thread Dawid Wysakowicz
+1 (binding) - signatures & checksums OK - checked changed licenses from 1.15.1 - PR OK - no excessive or binary files in the source distribution Best, Dawid On 19.08.2022 10:30, Xingbo Huang wrote: +1 (non-binding) - verify signatures and checksums - no binaries found in source archive -

Re: [VOTE] FLIP-227: Support overdraft buffer

2022-05-26 Thread Dawid Wysakowicz
+1 (binding) On Thu, 26 May 2022, 11:21 Piotr Nowojski, wrote: > Yes, it will be a good improvement :) > > +1 (binding) > > Piotrek > > czw., 26 maj 2022 o 10:26 Anton Kalashnikov > napisał(a): > > > Hi. > > > > Thanks Fanrui for this FLIP. I think it will be useful thing for us. > > > >

[jira] [Created] (FLINK-27489) Allow users to run dedicated tests in the CI

2022-05-04 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-27489: Summary: Allow users to run dedicated tests in the CI Key: FLINK-27489 URL: https://issues.apache.org/jira/browse/FLINK-27489 Project: Flink Issue

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-05-04 Thread Dawid Wysakowicz
enabling the Unaligned Checkpoint, it uses up to maxOverdraftBuffersPerPartition buffers. - If disabling the UC, it doesn't use the overdraft buffer. - Do you think it's ok? - Actually, we added the checkAvailable logic for LegacySource in our inte

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-04-26 Thread Dawid Wysakowicz
ood something. My perspective would be: Top A: Yes Top B: Yes (but disable watermark trigger for split alignment) Top C: No Top D: b) Best, Sebastian On Tue, Apr 26, 2022 at 9:55 AM Dawid Wysakowicz wrote: @Arvid: While I also like Becket's capability ap

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-04-26 Thread Dawid Wysakowicz
e if it makes sense to include Kafka and Pulsar in the FLIP. For me, this is rather immediate follow-up work. (could be in the same umbrella ticket) Best, Arvid On Mon, Apr 25, 2022 at 12:52 PM Dawid Wysakowicz wrote: a) "MySourceReader implements SourceReader, WithSplitsAlignment&quo

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-04-25 Thread Dawid Wysakowicz
implements SourceReader" and "MySplitReader implements SplitReader". I think the latest proposal according to Dawid would be: d) "MySourceReader implements SourceReader" and "MySplitReader implements AlignedSplitReader". I am fine with this API, although personally spea

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-04-25 Thread Dawid Wysakowicz
reness from Flink, we can add more such methods to the Source. What do you think? Thanks, Jiangjie (Becket) Qin On Fri, Apr 22, 2022 at 4:05 PM Dawid Wysakowicz wrote: @Konstantin: As part of this FLIP, the `AlignedSplitReader` interface (aka the stop & resume behavior) will be impleme

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-04-22 Thread Dawid Wysakowicz
tory with this FLIP. Cheers, Konstantin On Thu, Apr 21, 2022 at 12:36 PM Dawid Wysakowicz < dwysakow...@apache.org> wrote: To be explicit, having worked on it, I support it ;) I think we can start a vote thread soonish, as there are no concerns so far. Best, Dawid On 13/04/2022 11:2

Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-21 Thread Dawid Wysakowicz
+1 (binding), * checked licenses diff to my previous checks on rc2, this time everything seems ok * checked checksums, signatures, there are no binaries * compiled from sources * run a standalone cluster, clicked through the UI * run StateMachineExample took a savepoint in native format

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-04-21 Thread Dawid Wysakowicz
WatermarkOutputMulitplexer and report watermarks to the SourceOperator. There is a PoC for this FLIP [2], prototyped by Arvid Heise and revised and reworked by Dawid Wysakowicz (He did most of the work.) and me. The changes are backwards compatible in a way that if affected components do

[jira] [Created] (FLINK-27319) Duplicated "-t" option for savepoint format and deployment target

2022-04-20 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-27319: Summary: Duplicated "-t" option for savepoint format and deployment target Key: FLINK-27319 URL: https://issues.apache.org/jira/browse/FLINK-27319

Re: [VOTE] Release 1.15.0, release candidate #2

2022-04-13 Thread Dawid Wysakowicz
-1, I checked new modules and modules with updated dependencies for proper listing LICENSE files. I found a number of issues, of which one justifies cancelling the rc, imo. Blocker: * https://issues.apache.org/jira/browse/FLINK-27231 Minor: *

[jira] [Created] (FLINK-27233) Unnecessary entries in connector-elasticsearch7 in NOTICE file

2022-04-13 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-27233: Summary: Unnecessary entries in connector-elasticsearch7 in NOTICE file Key: FLINK-27233 URL: https://issues.apache.org/jira/browse/FLINK-27233 Project

[jira] [Created] (FLINK-27231) SQL pulsar connector lists dependencies under wrong license

2022-04-13 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-27231: Summary: SQL pulsar connector lists dependencies under wrong license Key: FLINK-27231 URL: https://issues.apache.org/jira/browse/FLINK-27231 Project: Flink

[jira] [Created] (FLINK-27230) Unnecessary entries in connector-kinesis NOTICE file

2022-04-13 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-27230: Summary: Unnecessary entries in connector-kinesis NOTICE file Key: FLINK-27230 URL: https://issues.apache.org/jira/browse/FLINK-27230 Project: Flink

[jira] [Created] (FLINK-27229) Cassandra overrides netty version in tests

2022-04-13 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-27229: Summary: Cassandra overrides netty version in tests Key: FLINK-27229 URL: https://issues.apache.org/jira/browse/FLINK-27229 Project: Flink Issue

  1   2   3   4   5   6   7   8   9   10   >