Re: [DISCUSS] Support multiple ordering fields

2021-01-19 Thread Danny Chan
> Wondering if we should just take a bunch of payload configs and deprecate these flags I have the same feeling, there are already so many config options in Hoodie, the maintain work for developers or users is hard. Vinoth Chandar 于2021年1月18日周一 下午11:40写道: > +1 as well. > > Slightly orthogonal p

?????? [DISCUSS] Support multiple ordering fields

2021-01-19 Thread ??????
+1?? Currently we have encountered such scenarios and look forward to supporting --  -- ??: "dev"

[DISCUSS] Rethink the abstraction of current client

2021-01-19 Thread vino yang
Hi guys, *I open this thread to discuss if we can separate the attributes and behaviors of HoodieTable, and rethink the abstraction of the client.* Currently, in the hudi-client-common module, there is a HoodieTable class, which contains a set of attributes and behaviors. For different engines, i

Re: [DISCUSS] Support multiple ordering fields

2021-01-19 Thread Pratyaksh Sharma
Hi We can use transformer to have a combination of multiple ordering fields. However custom Comparable implementation is not possible in that case. So overall a +1 from my side as well. On Tue, Jan 19, 2021 at 1:58 PM 刘金辉 <965147...@qq.com> wrote: > +1, Currently we have encountered such scenari

Re: [DISCUSS] Rethink the abstraction of current client

2021-01-19 Thread Danny Chan
> It contains three components: - Two objects: a table, a batch of records; For the Spark client, it is true because no matter Spark or Spark streaming engine, they write as batches, but things are different for pure streaming engines like Flink, Flink writes per-record, it does not accumulate

Re: 0.7.0 Release planning

2021-01-19 Thread Vinoth Chandar
Folks, travis has been very slow and now or builds have stopped running altogether. Any one with some ideas? On Mon, Jan 18, 2021 at 7:37 AM Vinoth Chandar wrote: > Winding down. > We have landed two major fixes based on testing > > 1. https://github.com/apache/hudi/pull/2441#issuecomment-76174

Re: 0.7.0 Release planning

2021-01-19 Thread Vinoth Chandar
Checked a few other apache projects. All of them facing the same. Release branch has to wait further I guess, until travis comes back Will keep this thread posted On Tue, Jan 19, 2021 at 7:32 PM Vinoth Chandar wrote: > Folks, travis has been very slow and now or builds have stopped running > al

Re: [DISCUSS] Rethink the abstraction of current client

2021-01-19 Thread vino yang
>> For the Spark client, it is true because no matter Spark or Spark streaming engine, they write as batches, but things are different for pure streaming engines like Flink, Flink writes per-record, it does not accumulate buffers. Yes, what I mean about the "batch" is not about the behavior or mec