Thank you for the feedbacks. I'm also in favor of simplifying the delta streamer's usage.
@vinoth, building on what you said, would it be simpler to merge the ordering field config with the payload class config? i.e., to remove both --source-ordering-field and payload ordering field config, then define the ordering field within the payload class. People who use fields other than `ts` will need to supply a payload class and override it. People who already supply a payload class will just override it in the existing custom class. WDYT? On Tue, Jan 19, 2021 at 10:40 AM Pratyaksh Sharma <pratyaks...@gmail.com> wrote: > Hi > > We can use transformer to have a combination of multiple ordering fields. > However custom Comparable implementation is not possible in that case. So > overall a +1 from my side as well. > > On Tue, Jan 19, 2021 at 1:58 PM 刘金辉 <965147...@qq.com> wrote: > > > +1， Currently we have encountered such scenarios and look forward to > > supporting > > > > > > > > > > ------------------ 原始邮件 ------------------ > > 发件人: > > "dev" > > < > > danny0...@apache.org>; > > 发送时间: 2021年1月19日(星期二) 下午4:25 > > 收件人: "dev"<dev@hudi.apache.org>; > > > > 主题: Re: [DISCUSS] Support multiple ordering fields > > > > > > > > > Wondering if we should just take a bunch of payload configs and > > deprecate > > these flags > > > > I have the same feeling, there are already so many config options in > > Hoodie, the maintain work for developers or users is hard. > > > > Vinoth Chandar <vin...@apache.org> 于2021年1月18日周一 下午11:40写道： > > > > > +1 as well. > > > > > > Slightly orthogonal point. > > > > > > Wondering if we should just take a bunch of payload configs and > > deprecate > > > these flags? > > > What I mean is : --source-ordering-field ultimately is used by the > > > HoodieAvroPayload class (or its family). > > > Our utilities expose these as flags for convenience, but its just > > more work > > > to maintain. > > > > > > On Mon, Jan 18, 2021 at 7:09 AM vino yang <yanghua1...@gmail.com > > > > wrote: > > > > > > > +1, > > > > > > > > We have found that such flexibility is needed in some > scenarios. > > > > > > > > Best, > > > > Vino > > > > > > > > Raymond Xu <xu.shiyan.raym...@gmail.com> 于2021年1月17日周日 > > 上午3:38写道： > > > > > > > > > Just want to discuss a small improvement for setting > > ordering fields. > > > > > For > > > > > - property `hoodie.payload.ordering.field` and > > > > > - deltastreamer --source-ordering-field > > > > > I think it can be useful to support multiple fields > > (configured via a > > > > > comma-separated list) to determine the order in some > cases. > > This would > > > > need > > > > > another config to set the Comparable implementation, say > > > > > hoodie.payload.ordering.comparable.class to allow custom > > logic for > > > doing > > > > > comparison. > > > > > > > > > > Any suggestions? Thanks. > > > > > > > > > > > > >
