Re: DISCUSS code, config, design walk through sessions

+1 On Mon, Jul 6, 2020 at 9:27 AM vbal...@apache.org wrote: > +1. > On Monday, July 6, 2020, 09:11:47 AM PDT, Bhavani Sudha < > bhavanisud...@gmail.com> wrote: > > +1 this is a great idea! > > On Mon, Jul 6, 2020 at 7:54 AM vino yang wrote: > > > +1 > > > > Adam Feldman 于2020年7月6日周一

Re: DISCUSS code, config, design walk through sessions

+1. On Monday, July 6, 2020, 09:11:47 AM PDT, Bhavani Sudha wrote: +1 this is a great idea! On Mon, Jul 6, 2020 at 7:54 AM vino yang wrote: > +1 > > Adam Feldman 于2020年7月6日周一 下午9:55写道： > > > Interested > > > > On Mon, Jul 6, 2020, 08:29 Sivabalan wrote: > > > > > +1 for sure > > >

Re: [DISCUSS] Make delete marker configurable?

+1 as well. Thanks Raymond for explaining. On Sun, Jun 28, 2020 at 11:35 AM Shiyan Xu wrote: > Hi Sudha, the delete marker being configurable can give more flexibility to > users when process delete events; they can check any bool field they may > have on their own schema. > > On Sat, Jun 27,

Re: DISCUSS code, config, design walk through sessions

+1 this is a great idea! On Mon, Jul 6, 2020 at 7:54 AM vino yang wrote: > +1 > > Adam Feldman 于2020年7月6日周一 下午9:55写道： > > > Interested > > > > On Mon, Jul 6, 2020, 08:29 Sivabalan wrote: > > > > > +1 for sure > > > > > > On Mon, Jul 6, 2020 at 4:42 AM Gurudatt Kulkarni > > > wrote: > > > > >

Re: DISCUSS code, config, design walk through sessions

+1 Adam Feldman 于2020年7月6日周一 下午9:55写道： > Interested > > On Mon, Jul 6, 2020, 08:29 Sivabalan wrote: > > > +1 for sure > > > > On Mon, Jul 6, 2020 at 4:42 AM Gurudatt Kulkarni > > wrote: > > > > > +1 > > > Really a great idea. Will help in understanding the project better. > > > > > > On Mon,

Re: DISCUSS code, config, design walk through sessions

Interested On Mon, Jul 6, 2020, 08:29 Sivabalan wrote: > +1 for sure > > On Mon, Jul 6, 2020 at 4:42 AM Gurudatt Kulkarni > wrote: > > > +1 > > Really a great idea. Will help in understanding the project better. > > > > On Mon, Jul 6, 2020 at 1:35 PM Pratyaksh Sharma > > wrote: > > > > > This

Re: DISCUSS code, config, design walk through sessions

+1 for sure On Mon, Jul 6, 2020 at 4:42 AM Gurudatt Kulkarni wrote: > +1 > Really a great idea. Will help in understanding the project better. > > On Mon, Jul 6, 2020 at 1:35 PM Pratyaksh Sharma > wrote: > > > This is a great idea and really helpful one. > > > > On Mon, Jul 6, 2020 at 1:09 PM

Re: DISCUSS code, config, design walk through sessions

+1 Really a great idea. Will help in understanding the project better. On Mon, Jul 6, 2020 at 1:35 PM Pratyaksh Sharma wrote: > This is a great idea and really helpful one. > > On Mon, Jul 6, 2020 at 1:09 PM wrote: > > > +1 > > It can also attract more partners to join us. > > > > > > > > On

Re: DISCUSS code, config, design walk through sessions

This is a great idea and really helpful one. On Mon, Jul 6, 2020 at 1:09 PM wrote: > +1 > It can also attract more partners to join us. > > > > On 07/06/2020 15:34, Ranganath Tirumala wrote: > +1 > > On Mon, 6 Jul 2020 at 16:59, David Sheard < > david.she...@datarefactory.com.au> > wrote: > > >

Re: DISCUSS code, config, design walk through sessions

+1 It can also attract more partners to join us. On 07/06/2020 15:34, Ranganath Tirumala wrote: +1 On Mon, 6 Jul 2020 at 16:59, David Sheard wrote: > Perfect > > On Mon, 6 Jul. 2020, 1:30 pm Vinoth Chandar, wrote: > > > Hi all, > > > > As we scale the community, its important that more of

Re: DISCUSS code, config, design walk through sessions

+1 On Mon, 6 Jul 2020 at 16:59, David Sheard wrote: > Perfect > > On Mon, 6 Jul. 2020, 1:30 pm Vinoth Chandar, wrote: > > > Hi all, > > > > As we scale the community, its important that more of us are able to help > > users, users becoming contributors. > > > > In the past, we have drafted

Re: DISCUSS code, config, design walk through sessions

Perfect On Mon, 6 Jul. 2020, 1:30 pm Vinoth Chandar, wrote: > Hi all, > > As we scale the community, its important that more of us are able to help > users, users becoming contributors. > > In the past, we have drafted faqs, trouble shooting guides. But I feel > sometimes, more hands on walk

Re: DISCUSS code, config, design walk through sessions

+1 It will be really helpful On Mon, 6 Jul 2020 at 11:53 AM, Shahida Khan wrote: > +1. > > *Regards,* > *Shahida R. Khan* > > > > > On Mon, 6 Jul 2020 at 09:46, Gary Li wrote: > > > +1. Technical deep dive will be very helpful. > > > > On Sun, Jul 5,

Re: DISCUSS code, config, design walk through sessions

+1. *Regards,* *Shahida R. Khan* On Mon, 6 Jul 2020 at 09:46, Gary Li wrote: > +1. Technical deep dive will be very helpful. > > On Sun, Jul 5, 2020 at 8:30 PM Vinoth Chandar wrote: > > > Hi all, > > > > As we scale the community, its important

Re: DISCUSS code, config, design walk through sessions

+1. Technical deep dive will be very helpful. On Sun, Jul 5, 2020 at 8:30 PM Vinoth Chandar wrote: > Hi all, > > As we scale the community, its important that more of us are able to help > users, users becoming contributors. > > In the past, we have drafted faqs, trouble shooting guides. But I

Re: [DISCUSS] Make delete marker configurable?

+1  Sent from Yahoo Mail for iPhone On Monday, June 29, 2020, 5:34 PM, Vinoth Chandar wrote: +1 as well. (sorry , for jumping in late) On Sun, Jun 28, 2020 at 11:36 AM Shiyan Xu wrote: > Thanks for the +1. Filed https://issues.apache.org/jira/browse/HUDI-1058 > > On Sat, Jun 27, 2020 at

Re: [DISCUSS] Make delete marker configurable?

+1 as well. (sorry , for jumping in late) On Sun, Jun 28, 2020 at 11:36 AM Shiyan Xu wrote: > Thanks for the +1. Filed https://issues.apache.org/jira/browse/HUDI-1058 > > On Sat, Jun 27, 2020 at 11:34 PM Pratyaksh Sharma > wrote: > > > The suggestion looks good to me as well. > > > > On Sun,

Re: [DISCUSS] Make delete marker configurable?

Thanks for the +1. Filed https://issues.apache.org/jira/browse/HUDI-1058 On Sat, Jun 27, 2020 at 11:34 PM Pratyaksh Sharma wrote: > The suggestion looks good to me as well. > > On Sun, Jun 28, 2020 at 8:17 AM Sivabalan wrote: > > > +1, I just left it as a todo for future patch when I worked on

Re: [DISCUSS] Make delete marker configurable?

Hi Sudha, the delete marker being configurable can give more flexibility to users when process delete events; they can check any bool field they may have on their own schema. On Sat, Jun 27, 2020 at 5:32 PM Bhavani Sudha wrote: > Hi Raymond, > > I am trying to understand the use case . Can you

Re: [DISCUSS] Make delete marker configurable?

The suggestion looks good to me as well. On Sun, Jun 28, 2020 at 8:17 AM Sivabalan wrote: > +1, I just left it as a todo for future patch when I worked on it. > > On Sat, Jun 27, 2020 at 8:32 PM Bhavani Sudha > wrote: > > > Hi Raymond, > > > > I am trying to understand the use case . Can you

Re: [DISCUSS] Make delete marker configurable?

+1, I just left it as a todo for future patch when I worked on it. On Sat, Jun 27, 2020 at 8:32 PM Bhavani Sudha wrote: > Hi Raymond, > > I am trying to understand the use case . Can you please provide more > context on what problem this addresses ? > > > Thanks, > Sudha > > On Fri, Jun 26,

Re: [DISCUSS] Make delete marker configurable?

Hi Raymond, I am trying to understand the use case . Can you please provide more context on what problem this addresses ? Thanks, Sudha On Fri, Jun 26, 2020 at 9:02 PM Shiyan Xu wrote: > Hi all, > > A small suggestion: as delta streamer relies on `_hoodie_is_deleted` to do > hard delete,

Re: [DISCUSS] Introduce a write committed callback hook

This is a great discussion! thanks! On Mon, Jun 22, 2020 at 6:33 PM vino yang wrote: > Hi everyone, > > Thanks for sharing your thoughts. > > We have created a Jira issue to track this work.[1] > > Best, > Vino > > [1]: https://issues.apache.org/jira/browse/HUDI-1037 > > Vinoth Chandar

Re: [DISCUSS] Introduce a write committed callback hook

Hi everyone, Thanks for sharing your thoughts. We have created a Jira issue to track this work.[1] Best, Vino [1]: https://issues.apache.org/jira/browse/HUDI-1037 Vinoth Chandar 于2020年6月23日周二 上午6:38写道： > Great, looks like a JIRA is in order? :), given we all agree > enthusiastically > > On

Re: [DISCUSS] Introduce a write committed callback hook

Great, looks like a JIRA is in order? :), given we all agree enthusiastically On Sun, Jun 21, 2020 at 8:10 PM Gary Li wrote: > +1. > That would be great to have a communication mechanism between downstream > CDC applications chain. > e.g. A->B->C->D. Right now I am using the commit timestamp to

Re: [DISCUSS] Publishing benchmarks for releases

+1 as well On Mon, Jun 22, 2020 at 4:19 AM David Sheard < david.she...@datarefactory.com.au> wrote: > Like the idea > > On Mon, 22 Jun. 2020, 9:56 am Sivabalan, wrote: > > > Hey folks, > > Is it a common practise to publish benchmarks for releases? I have > put > > up an initial PR

Re: [DISCUSS] Publishing benchmarks for releases

Like the idea On Mon, 22 Jun. 2020, 9:56 am Sivabalan, wrote: > Hey folks, > Is it a common practise to publish benchmarks for releases? I have put > up an initial PR to add jmh > benchmark support to a couple of Hudi operations. If the community

Re: [DISCUSS] Publishing benchmarks for releases

@Vinoth Chandar : yes, I actually proposed the idea on bumping at lucene's benchmark page only :) On Mon, Jun 22, 2020 at 3:23 AM vbal...@apache.org wrote: > > +1 on adding benchmarks.On Sunday, June 21, 2020, 11:18:05 PM PDT, > Mario de Sá Vera wrote: > > +1 for performance reports > >

Re: [DISCUSS] Publishing benchmarks for releases

+1 on adding benchmarks.On Sunday, June 21, 2020, 11:18:05 PM PDT, Mario de Sá Vera wrote: +1 for performance reports On Mon, 22 Jun 2020, 02:41 vino yang, wrote: > +1 as well, > > it would be helpful to measure the performance between different versions. > > Shiyan Xu

Re: [DISCUSS] Publishing benchmarks for releases

+1 for performance reports On Mon, 22 Jun 2020, 02:41 vino yang, wrote: > +1 as well, > > it would be helpful to measure the performance between different versions. > > Shiyan Xu 于2020年6月22日周一 上午8:37写道： > > > +1 definitely useful info. > > > > On Sun, Jun 21, 2020 at 4:56 PM Sivabalan wrote:

Re: [DISCUSS] Introduce a write committed callback hook

+1. That would be great to have a communication mechanism between downstream CDC applications chain. e.g. A->B->C->D. Right now I am using the commit timestamp to identify whether there is a new commit came in. But if I need to recompute app B, it’s difficult for C and D to aware they have to

Re: [DISCUSS] Publishing benchmarks for releases

Lucene has nightly runs even https://home.apache.org/~mikemccand/lucenebench/ We can do something like this? In any case, raising a Jira under performance component seems like a good idea? On Sun, Jun 21, 2020 at 6:41 PM vino yang wrote: > +1 as well, > > it would be helpful to measure the

Re: [DISCUSS] Publishing benchmarks for releases

+1 as well, it would be helpful to measure the performance between different versions. Shiyan Xu 于2020年6月22日周一 上午8:37写道： > +1 definitely useful info. > > On Sun, Jun 21, 2020 at 4:56 PM Sivabalan wrote: > > > Hey folks, > > Is it a common practise to publish benchmarks for releases? I

Re: [DISCUSS] Introduce a write committed callback hook

+1. a great feature. Sivabalan 于2020年6月22日周一 上午7:50写道： > +1. would be a nice addition. > > On Sun, Jun 21, 2020 at 12:02 PM vbal...@apache.org > wrote: > > > > > +1. This would be a really good feature to have when building dependent > > ETL pipelines. > > > > On Friday, June 19, 2020,

Re: [DISCUSS] Publishing benchmarks for releases

+1 definitely useful info. On Sun, Jun 21, 2020 at 4:56 PM Sivabalan wrote: > Hey folks, > Is it a common practise to publish benchmarks for releases? I have put > up an initial PR to add jmh > benchmark support to a couple of Hudi operations. If

Re: [DISCUSS] Introduce a write committed callback hook

+1. would be a nice addition. On Sun, Jun 21, 2020 at 12:02 PM vbal...@apache.org wrote: > > +1. This would be a really good feature to have when building dependent > ETL pipelines. > > On Friday, June 19, 2020, 05:13:45 PM PDT, vino yang < > vinoy...@apache.org> wrote: > > Hi all, > >

Re: [DISCUSS] Introduce a write committed callback hook

+1. This would be a really good feature to have when building dependent ETL pipelines. On Friday, June 19, 2020, 05:13:45 PM PDT, vino yang wrote: Hi all, Currently, we have a need to incrementally process and build a new table based on an original hoodie table. We expect that

Re: [DISCUSS] Introduce a write committed callback hook

+1. It is a great complement to the pull model; helpful to fan-out scenarios On Sun, Jun 21, 2020 at 8:07 AM Bhavani Sudha wrote: > +1 . I think this is a valid use case and would be useful in general. > > On Sun, Jun 21, 2020 at 7:11 AM Vinoth Chandar wrote: > > > +1 as well > > > > > We

Re: [DISCUSS] Introduce a write committed callback hook

+1 . I think this is a valid use case and would be useful in general. On Sun, Jun 21, 2020 at 7:11 AM Vinoth Chandar wrote: > +1 as well > > > We expect to introduce a proactive notification(event callback) > mechanism. For example, a hook can be introduced after a successful commit. > > This

Re: [DISCUSS] Regarding nightly builds

+1 as well, Currently, I am waiting for hudi-test-suite to be merged into the master branch, so that when we have a new PR merged into the master branch, this will cause the "hudi-test-suite" that is also on the master branch to be triggered on Azure Pipeline " easier. Sharing more information

Re: [DISCUSS] Regarding nightly builds

Hi Sudha, Thanks for getting this kicked off.. +1 on a new nightly build process.. This will help us more easily make the bleeding edge testable.. My initial thoughts here are - Figure out a way to get Azure Pipelines enabled for Hudi - Setup the nightly there (this will also help us

Re: [DISCUSS] Introduce a write committed callback hook

+1 as well > We expect to introduce a proactive notification(event callback) mechanism. For example, a hook can be introduced after a successful commit. This would be very useful. We could write to a variety of event bus-es and notify new data arrival. On Sat, Jun 20, 2020 at 2:51 AM

Re: [DISCUSS] Introduce a write committed callback hook

+1 for this, I think this is a feature worth doing. Think about it in the filed of offline computing, data changes happens hourly or daily, if there is no a notification mechanism to inform the downstream, then the tasks downstream will keeping running all the day along, but the time really

Re: [DISCUSS] querying commit metadata from spark DataSource

Yes, tickets linked. On Thu, Jun 11, 2020 at 10:50 AM Vinoth Chandar wrote: > Thanks Raymond! > > yes.. we can make this a config and leave it to the user to decide if they > want to use a global table for all their hudi tables (or) keep > one error table for each hudi table.. > > For this

Re: [DISCUSS] querying commit metadata from spark DataSource

Thanks Raymond! yes.. we can make this a config and leave it to the user to decide if they want to use a global table for all their hudi tables (or) keep one error table for each hudi table.. For this effort, does it make sense to take a dependency on the multi-writer jira HUDI-944, that liwei

Re: [DISCUSS] querying commit metadata from spark DataSource

Yes, Vinoth, it does go a bit too far with first class support on these data. A global error table can do the job easily. As we discussed yesterday, parallel local error tables with `_errors` suffix could also benefit for some scenarios, like different product teams manage their own tables or in

Re: [DISCUSS] Write failed records

Thanks! Will review and get back to you On Tue, Jun 2, 2020 at 10:37 AM Shiyan Xu wrote: > Thank you for the feedback, Vinoth. Agreed with your points. Also created a > small RFC for easy alignment on the changes > >

Re: [DISCUSS] querying commit metadata from spark DataSource

Hi Raymond, I am not sure generalizing this to all metadata like - errors and metrics - would be a good idea. We can certainly implement logging errors to a common errors hudi table, with a certain schema. But these can be just regular “hudi” format tables. Unlike the timeline metadata, these

Re: [DISCUSS] querying commit metadata from spark DataSource

I also encountered use cases where I'd like to programmatically query metadata. +1 on the idea of format(“hudi-timeline”) I also feel that the metadata can be extended further to include more info like, errors, metrics/write statistics, etc. Like the newly proposed error handling, we could also

Re: [DISCUSS] querying commit metadata from spark DataSource

Got it. I'll look into implementation choices for creating a new data source. Appreciate all the feedback. On Mon, Jun 1, 2020 at 7:53 PM Vinoth Chandar wrote: > >Is it to separate data and metadata access? > Correct. We already have modes for querying data using format("hudi"). I > feel it

Re: [DISCUSS] querying commit metadata from spark DataSource

>Is it to separate data and metadata access? Correct. We already have modes for querying data using format("hudi"). I feel it will get very confusing to mix data and metadata in the same source.. for e.g a lot of options we support for data may not even make sense for the TimelineRelation. >This

Re: [DISCUSS] querying commit metadata from spark DataSource

Thanks for the feedback. What is the advantage of doing spark.read.format(“hudi-timeline”).load(basepath) as opposed to doing new relation? Is it to separate data and metadata access? Are you looking for similar functionality as HoodieDatasourceHelpers? > This class seems like a list of static

Re: [DISCUSS] querying commit metadata from spark DataSource

Also please take a look at https://issues.apache.org/jira/browse/HUDI-309. This was an effort to make the timeline more generalized for querying (for a different purpose).. but good to revisit now.. On Sun, May 31, 2020 at 11:04 PM vbal...@apache.org wrote: > > I strongly recommend using a

Re: [DISCUSS] querying commit metadata from spark DataSource

I strongly recommend using a separate datasource relation (option 1) to query timeline. It is elegant and fits well with spark APIs. Thanks.Balaji.VOn Saturday, May 30, 2020, 01:18:45 PM PDT, Vinoth Chandar wrote: Hi satish, Are you looking for similar functionality as

Re: [DISCUSS] querying commit metadata from spark DataSource

Hi satish, Are you looking for similar functionality as HoodieDatasourceHelpers? We have historically relied on cli to inspect the table, which does not lend it self well to programmatic access.. overall in like option 1 - allowing the timeline to be queryable with a standard schema does seem

Re: [DISCUSS] Write failed records

Hi Raymond, Thanks for starting this discussion. Agree on 1.. (we may also need some CLI support for inspecting bad/record and also code samples to consume them etc?) On 2, these place seem appropriate. We can figure it out, in more detail when we get to implementation? On 3. +1 on logs.. We

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

Hi Sudha, Thanks for the update. Did my best to review most of them. Will work on 1596, just needs a test. Thanks Vinoth On Sun, May 17, 2020 at 3:30 PM Bhavani Sudha wrote: > Hello all, > > I wanted to send a quick update on 0.5.3 readiness and code freeze. > There were few more candidate

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

Hello all, I wanted to send a quick update on 0.5.3 readiness and code freeze. There were few more candidate that requested to go in 0.5.3. Of those, we are still waiting on the following 5 PRs to be reviewed and landed. - #1633 HUDI-858 Allow

Re: [DISCUSS] Logos on project front page.

All green :) On Wed, May 13, 2020 at 12:54 PM Vinoth Chandar wrote: > https://svn.apache.org/repos/asf/comdev/project-logos/originals/ > Checked our logo svg in. This must be the last issue on the whimsy page. > > On Wed, May 13, 2020 at 8:23 AM Sivabalan wrote: > > > sorry, I am getting

Re: [DISCUSS] Logos on project front page.

https://svn.apache.org/repos/asf/comdev/project-logos/originals/ Checked our logo svg in. This must be the last issue on the whimsy page. On Wed, May 13, 2020 at 8:23 AM Sivabalan wrote: > sorry, I am getting access issues. I have pinged the details in the slack > channel asking for help. > > >

Re: [DISCUSS] Logos on project front page.

sorry, I am getting access issues. I have pinged the details in the slack channel asking for help. On Wed, May 13, 2020 at 10:59 AM Vinoth Chandar wrote: > https://github.com/apache/incubator-hudi/pull/1628 Fixed most of the > issues > raised (what I mentioned in the previous email) > > We

Re: [DISCUSS] Logos on project front page.

https://github.com/apache/incubator-hudi/pull/1628 Fixed most of the issues raised (what I mentioned in the previous email) We need to upload the logo, which I assume Siva you are taking care of ? On Wed, May 13, 2020 at 7:45 AM Vinoth Chandar wrote: >

Re: [DISCUSS] Logos on project front page.

https://whimsy.apache.org/pods/project/hudi We need to get this in order, looks like.. On Wed, May 13, 2020 at 5:24 AM Sivabalan wrote: > From what I could infer(from the attached link), PMC can decide whats best > for the project. (correct me if I am wrong), i.e. choose to have it in home >

Re: [DISCUSS] Logos on project front page.

>From what I could infer(from the attached link), PMC can decide whats best for the project. (correct me if I am wrong), i.e. choose to have it in home page or a sec page. In that case, we can leave it in front page as it would assist in gaining traction. Also, I see flink

Re: [DISCUSS] Logos on project front page.

I agree on following the best practices. Bhavani Sudha 于2020年5月13日周三 下午4:30写道： > +1 > > From the doc it seems like we can probably move them to secondary page. > > Thanks, > Sudha > > On Tue, May 12, 2020 at 11:56 PM vino yang wrote: > > > +1 to follow the best practices. > > > >

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

+1 Bhavani Sudha 于2020年5月13日周三 下午1:19写道： > Thank you all. I created a jira here - > https://jira.apache.org/jira/browse/HUDI-890 that tracks the list of > patches going into this release. So far I am able to cherry pick these > commits and get a successful local run of tests. > > Let us aim to

Re: [DISCUSS] Logos on project front page.

+1 >From the doc it seems like we can probably move them to secondary page. Thanks, Sudha On Tue, May 12, 2020 at 11:56 PM vino yang wrote: > +1 to follow the best practices. > > vbal...@apache.org 于2020年5月13日周三 上午10:31写道： > > > > > I agree on following the best practices. > > Balaji.VOn

Re: [DISCUSS] Logos on project front page.

+1 to follow the best practices. vbal...@apache.org 于2020年5月13日周三 上午10:31写道： > > I agree on following the best practices. > Balaji.VOn Tuesday, May 12, 2020, 06:52:59 PM PDT, Vinoth Chandar < > vin...@apache.org> wrote: > > Hello all, > > This was raised during the graduation discussion.

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

Thank you all. I created a jira here - https://jira.apache.org/jira/browse/HUDI-890 that tracks the list of patches going into this release. So far I am able to cherry pick these commits and get a successful local run of tests. Let us aim to code freeze by end of this week (May 16th EOD) for more

Re: [DISCUSS] Logos on project front page.

I agree on following the best practices. Balaji.VOn Tuesday, May 12, 2020, 06:52:59 PM PDT, Vinoth Chandar wrote: Hello all, This was raised during the graduation discussion. We have been referred to [1]. The doc ends saying. "These best practices for linking to outside pages on

Re: [DISCUSS] Why add unit tests for hudi-cli module

This is mainly a function test, not a parameter test. Also, ensure that all base commands are executed successfully.

Re: [DISCUSS] Why add unit tests for hudi-cli module

+1 People rely on CLI to operate on Hudi datasets. So having some tests there, would definitely be useful On Tue, May 12, 2020 at 12:39 PM Shiyan Xu wrote: > Hi, the tests in hudi-cli are more of functional tests. They are conducive > to verifying features in cli module are working. Though not

Re: [DISCUSS] Why add unit tests for hudi-cli module

Hi, the tests in hudi-cli are more of functional tests. They are conducive to verifying features in cli module are working. Though not covering all options, it is always better to have some assuring passing tests than none, isn't it? :) On Tue, May 12, 2020 at 8:31 AM hmantu wrote: > hi all, >

Re: [DISCUSS] Bug bash?

Checked that this is already tagged in the list Siva is putting up.. https://issues.apache.org/jira/issues/?jql=labels%20%3D%20bug-bash-0.6.0 On Fri, May 8, 2020 at 2:22 AM Pratyaksh Sharma wrote: > We can include this issue in our bug bash - >

Re: [DISCUSS] Insert Overwrite with snapshot isolation

Thanks, Satish! Will review! On Fri, May 8, 2020 at 4:38 PM Satish Kotha wrote: > Hello everyone, > > I started RFC here > > https://cwiki.apache.org/confluence/display/HUDI/RFC+-+18+Insert+Overwrite+API > . > Appreciate any feedback. > > Thanks > Satish > > On Tue, Apr 21, 2020 at 9:34 AM

Re: [DISCUSS] Insert Overwrite with snapshot isolation

Hello everyone, I started RFC here https://cwiki.apache.org/confluence/display/HUDI/RFC+-+18+Insert+Overwrite+API. Appreciate any feedback. Thanks Satish On Tue, Apr 21, 2020 at 9:34 AM nishith agarwal wrote: > +1, thanks for starting this effort Satish! > > -Nishith > > On Fri, Apr 17, 2020

Re: [DISCUSS] Bug bash?

We can include this issue in our bug bash - https://github.com/apache/incubator-hudi/issues/1599. On Fri, May 8, 2020 at 12:51 AM Pratyaksh Sharma wrote: > Missed this thread. Happy to volunteer in fixing as many bugs as possible. > :) > > On Thu, May 7, 2020 at 7:53 PM Sivabalan wrote: > >>

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

https://github.com/apache/incubator-hudi/commit/dbc9acd23a4eb208c7cd458bb3adaf54731d4145 On 2020/05/06 20:31:00, Vinoth Chandar wrote: > Hi Sudha,> > > +1 on the overall idea.. I tried to pick out

Re: [DISCUSS] Bug bash?

Missed this thread. Happy to volunteer in fixing as many bugs as possible. :) On Thu, May 7, 2020 at 7:53 PM Sivabalan wrote: > sure. thanks for the detailed pointers. Will work on it. > > On Thu, May 7, 2020 at 1:50 AM Vinoth Chandar wrote: > > > siva, That would be great. Next step is to put

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

Thanks for the responses. We will go ahead with 0.5.3 efforts. I was going to RM the 0.6.0 release. I ll focus on this one instead. If anyone else wants to drive the 0.6.0 please feel free to do so. Thanks, Sudha On Wed, May 6, 2020 at 10:55 PM vbal...@apache.org wrote: > +1 for releasing

Re: [DISCUSS] Bug bash?

sure. thanks for the detailed pointers. Will work on it. On Thu, May 7, 2020 at 1:50 AM Vinoth Chandar wrote: > siva, That would be great. Next step is to put together a bug list > > - Scour existing 0.6.0 tickets, nudge existing owners to see if they are > still actively driving this, else

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

+1 for releasing 0.5.3. Balaji.V On Wednesday, May 6, 2020, 10:36:54 PM PDT, Y Ethan Guo wrote: +1 On Wed, May 6, 2020 at 6:29 PM vino yang wrote: > +1 for 0.5.3 as well > > Nishith 于2020年5月7日周四 上午8:16写道： > > > +1 on the idea > > > > Sent from my iPhone > > > > > On May 6, 2020, at

Re: [DISCUSS] Bug bash?

siva, That would be great. Next step is to put together a bug list - Scour existing 0.6.0 tickets, nudge existing owners to see if they are still actively driving this, else unassign them - Look at mailing list, GH , slack for recently reported issues (file JIRAs if any slipped through the

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

+1 On Wed, May 6, 2020 at 6:29 PM vino yang wrote: > +1 for 0.5.3 as well > > Nishith 于2020年5月7日周四 上午8:16写道： > > > +1 on the idea > > > > Sent from my iPhone > > > > > On May 6, 2020, at 3:09 PM, Shiyan Xu > > wrote: > > > > > >

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

+1 for 0.5.3 as well Nishith 于2020年5月7日周四 上午8:16写道： > +1 on the idea > > Sent from my iPhone > > > On May 6, 2020, at 3:09 PM, Shiyan Xu > wrote: > > >

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

+1 for 0.5.3 as well On Wed, May 6, 2020 at 1:55 PM Sivabalan wrote: > sounds good Sudha. Let's have a good list of projects/features to be done > for 0.6.0 and not end up in a similar situation. I am ok to go with 0.5.3. > > On Wed, May 6, 2020 at 4:31 PM Vinoth Chandar wrote: > > > Hi Sudha,

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

sounds good Sudha. Let's have a good list of projects/features to be done for 0.6.0 and not end up in a similar situation. I am ok to go with 0.5.3. On Wed, May 6, 2020 at 4:31 PM Vinoth Chandar wrote: > Hi Sudha, > > +1 on the overall idea.. I tried to pick out few of these PRs that are > > -

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

Hi Sudha, +1 on the overall idea.. I tried to pick out few of these PRs that are - Small enough to apply easily - Have limited scope, fixing pointed problems - Have high impact on performance or usability [HUDI-799] Use appropriate FS when loading configs

Re: [DISCUSS] Return schema provider as optional?

I think discussions are going on in the PR itself.. Please chime in there as well if this suits you. On Sat, May 2, 2020 at 6:01 AM Shiyan Xu wrote: > Hi all, > > In case of reading schema-inferable source like parquet, when no new data > is found, then, if i understand correctly, no schema can

Re: [DISCUSS] moving blog from cwiki to website

Cool. Will get this done today. On Mon, May 4, 2020 at 11:02 AM Vinoth Chandar wrote: > Hi Prashant, > > We already have a site setup and apache hosting it. > See >

Re: [DISCUSS] moving blog from cwiki to website

Hi Prashant, We already have a site setup and apache hosting it. See https://github.com/apache/incubator-hudi/tree/asf-site for instructions for building locally and making changes etc. Like I mentioned before, it should be a simple matter of moving the posts in proper formatting to

Re: [DISCUSS] moving blog from cwiki to website

Hi Team, I surveyed several Apache projects and this is how they are blogging: 1. Use ASF's blogging platform. Hosted by ASF Example: https://blogs.apache.org/kafka/ 2. Aggregating links to blog posts posted elsewhere Example: https://docs.pinot.apache.org/community-1/blogs 3.

Re: [DISCUSS] Add Github and Twitter Widget on Hudi's official website

Thanks for everyone. I have filed a Jira issue: HUDI-861[1] to track this work. [1]: https://issues.apache.org/jira/browse/HUDI-861 leesf 于2020年5月3日周日 下午7:19写道： > +1 > > Vinoth Chandar 于2020年5月2日周六 上午12:05写道： > > > +1 these are the the little things that matter :) > > > > On Fri, May 1, 2020

Re: [DISCUSS] Add Github and Twitter Widget on Hudi's official website

+1 Vinoth Chandar 于2020年5月2日周六 上午12:05写道： > +1 these are the the little things that matter :) > > On Fri, May 1, 2020 at 2:28 AM wangxianghu wrote: > > > Hi vino, > > That’s a good idea, Adding the Github to Hudi’s official website will > make > > it more convenient to get Hudi's source code,

Re: [DISCUSS] Readiness for graduation to TLP

Great! Thanks everyone for chiming in. I follow up with next steps, including a formal vote by the community On Wed, Apr 29, 2020 at 6:16 PM hddong wrote: > +1 > > Thomas Weise 于2020年4月30日周四 上午2:58写道： > > > +1 > > > > On Wed, Apr 29, 2020 at 10:39 AM Luciano Resende > > wrote: > > > > > +1 >

Re: [DISCUSS] moving blog from cwiki to website

That’d be awesome! Thanks! On Fri, May 1, 2020 at 9:06 AM Prashant Wason wrote: > Hi Vinoth, > > Sure, I will prioritize this. Hope to have something by this weekend. > > Thanks > Prashant > > > On Wed, Apr 29, 2020 at 8:31 PM Vinoth Chandar wrote: > > > Hi Prashant, > > > > Have you started

Re: [DISCUSS] moving blog from cwiki to website

Hi Vinoth, Sure, I will prioritize this. Hope to have something by this weekend. Thanks Prashant On Wed, Apr 29, 2020 at 8:31 PM Vinoth Chandar wrote: > Hi Prashant, > > Have you started on this already? Any rough etas? > It might be good to have this in place soon so people can start

Re: [DISCUSS] Add Github and Twitter Widget on Hudi's official website

+1 these are the the little things that matter :) On Fri, May 1, 2020 at 2:28 AM wangxianghu wrote: > Hi vino, > That’s a good idea, Adding the Github to Hudi’s official website will make > it more convenient to get Hudi's source code, and since Twitter is one of > the most popular social

Re: [DISCUSS] Add Github and Twitter Widget on Hudi's official website

Hi vino, That’s a good idea, Adding the Github to Hudi’s official website will make it more convenient to get Hudi's source code, and since Twitter is one of the most popular social tools, It will surely help to strength the influence of Hudi ! Besides, as @tison said he picture is broken, use

Re: [DISCUSS] Add Github and Twitter Widget on Hudi's official website

Hi vino, I like the idea. Adding such widgets bring more visibility to our artifacts. The picture attached looks broken. You might use a link instead. Best, tison. vino yang 于2020年5月1日周五 下午1:56写道： > Hi guys, > > In order to further strengthen the influence of the Hudi community. I > suggest

<    1   2   3   4   5   6   7   8   9   10   >