Thanks Yunfeng, Let me try to answer your question :)
> 1. Would it be better to have all XXXPartitionStream classes implement ProcessConfigurable, instead of defining both XXXPartitionStream and ProcessConfigurableAndXXXPartitionStream? I wonder whether users would need to operate on a non-configurable PartitionStream. I thought about this for a while and decided to separate DataStream from ProcessConfigurable. At the core of this is that streams and configurations are completely orthogonal concepts, and configuration is only responsible for the `Process`, not the `Stream`. This is why only the ` process/connectAndProcess` returns configurable stream, but partitioning like `KeyBy` returns a pure DataStream. This may also answer your second question in passing. > Apart from the detailed withConfigFoo(foo)/withConfigBar(bar) methods, would it be better to also add a general withConfig(configKey, configValue) method to the ProcessConfigurable interface? Adding a method for each configuration might harm the readability and compatibility of configurations. Sorry, I may not fully understand this question. ProcessConfigurable simply refers to the configuration of the Process, which can have the name, parallelism, etc of the process. It's not actually the Configuratiion(Contains a lot of ConfigOptions) that we usually talk about, but more like `SingleOutputStreamOperator` in DataStream V1. Best regards, Weijie Xuannan Su <suxuanna...@gmail.com> 于2024年1月29日周一 18:45写道： > Hi Weijie, > > Thanks for the FLIP! I have a few questions regarding the FLIP. > > 1. +1 to only use XXXParititionStream if users only need to use the > configurable PartitionStream. If there are use cases for both, > perhaps we could use `ProcessConfigurableNonKeyedPartitionStream` or > `ConfigurableNonKeyedPartitionStream` for simplicity. > > 2. Should we allow users to set custom configurations through the > `ProcessConfigurable` interface and access these configurations in the > `ProcessFunction` via `RuntimeContext`? I believe it would be useful > for process function developers to be able to define custom > configurations. > > 3. How can users define custom metrics within the `ProcessFunction`? > Will there be a method like `getMetricGroup` available in the > `RuntimeContext`? > > Best, > Xuannan > > > On Fri, Jan 26, 2024 at 2:38 PM Yunfeng Zhou > <flink.zhouyunf...@gmail.com> wrote: > > > > Hi Weijie, > > > > Thanks for introducing this FLIP! I have a few questions about the > > designs proposed. > > > > 1. Would it be better to have all XXXPartitionStream classes implement > > ProcessConfigurable, instead of defining both XXXPartitionStream and > > ProcessConfigurableAndXXXPartitionStream? I wonder whether users would > > need to operate on a non-configurable PartitionStream. > > > > 2. The name "ProcessConfigurable" seems a little ambiguous to me. Will > > there be classes other than XXXPartitionStream that implement this > > interface? Will "Process" be accurate enough to describe > > PartitionStream and those classes? > > > > 3. Apart from the detailed withConfigFoo(foo)/withConfigBar(bar) > > methods, would it be better to also add a general > > withConfig(configKey, configValue) method to the ProcessConfigurable > > interface? Adding a method for each configuration might harm the > > readability and compatibility of configurations. > > > > Looking forward to your response. > > > > Best regards, > > Yunfeng Zhou > > > > On Tue, Dec 26, 2023 at 2:47 PM weijie guo <guoweijieres...@gmail.com> > wrote: > > > > > > Hi devs, > > > > > > > > > I'd like to start a discussion about FLIP-410: Config, Context and > > > Processing Timer Service of DataStream API V2 [1]. This is the second > > > sub-FLIP of DataStream API V2. > > > > > > > > > In FLIP-409 [2], we have defined the most basic primitive of > > > DataStream V2. On this basis, this FLIP will further answer several > > > important questions closely related to it: > > > > > > 1. > > > How to configure the processing over the datastreams, such as > > > setting the parallelism. > > > 2. > > > How to get access to the runtime contextual information and > > > services from inside the process functions. > > > 3. How to work with processing-time timers. > > > > > > You can find more details in this FLIP. Its relationship with other > > > sub-FLIPs can be found in the umbrella FLIP > > > [3]. > > > > > > > > > Looking forward to hearing from you, thanks! > > > > > > > > > Best regards, > > > > > > Weijie > > > > > > > > > [1] > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-410%3A++Config%2C+Context+and+Processing+Timer+Service+of+DataStream+API+V2 > > > > > > [2] > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-409%3A+DataStream+V2+Building+Blocks%3A+DataStream%2C+Partitioning+and+ProcessFunction > > > > > > [3] > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-408%3A+%5BUmbrella%5D+Introduce+DataStream+API+V2 >
