Re: Tablet Server with almost 1TB of WALs (and large number of open files)

2020-04-01 Thread Franco VENTURI
upply the fix for KUDU-3002 > > as a > > patch for your release of Kudu. Looking at the fix, it's not super > > invasive and should be pretty easy to backport into an older Kudu > > release. Have you talked to them about that? > > > > O

Re: Tablet Server with almost 1TB of WALs (and large number of open files)

2020-03-31 Thread Franco VENTURI
Adar, Andrew, thanks for your detailed and prompt replies. "Fortunately" (for your questions) we have another TS whose WALs disk is currently about 80% full (and three more whose WALs disk is above > 50%), and I suspect that it will be the next one we'll have to restart in a few nights. On

Tablet Server with almost 1TB of WALs (and large number of open files)

2020-03-28 Thread Franco VENTURI
this in the Kudu users mailing list, in case someone has ideas of what could cause this behavior and how to address it, and to find out if anyone else here has noticed something similar on their Kudu clusters, or if it is just peculiar to our configuration and type of load. Thanks in advance, Franco

Re: Kudu table api

2019-03-23 Thread Franco VENTURI
An approach to speed up counting the number of rows using the Kudu client API could first retrieve the list of KuduScanTokens for the scan below and then use several worker threads in parallel to add up the number of rows. I implemented it in Java this afternoon and it took about 13s to go

Re: Efficient way of computing max(PK) in Kudu

2017-12-14 Thread Franco Venturi
for your usecase, but hopefully with some additional features Kudu will eventually get there. I'd be very interested to hear your results if your pursue the binary search solution. - Dan On Wed, Dec 13, 2017 at 6:54 PM, Franco Venturi < fvent...@comcast.net > wrote: Scenario: Kudu table with a

Re: Error message: 'Tried to update clock beyond the max. error.'

2017-11-02 Thread Franco Venturi
Wed, Nov 1, 2017 at 6:31 PM, Franco Venturi < fvent...@comcast.net > wrote: >From 'tablet_bootsratp.cc': 1030 14:29:37.324306 60682 tablet_bootstrap.cc:884] Check failed: _s.ok() Bad status: Invalid argument: Tried to update clock beyond the max. error. Franco From: "To

Re: Error message: 'Tried to update clock beyond the max. error.'

2017-11-01 Thread Franco Venturi
du.apache.org Sent: Wednesday, November 1, 2017 8:00:09 PM Subject: Re: Error message: 'Tried to update clock beyond the max. error.' What's the full log line where you're seeing this crash? Is it coming from tablet_bootstrap.cc, raft_consensus.cc, or elsewhere? -Todd 2017-11-01 15:45 GMT-0

Re: Error message: 'Tried to update clock beyond the max. error.'

2017-11-01 Thread Franco Venturi
Our version is kudu 1.5.0-cdh5.13.0. Franco

Error message: 'Tried to update clock beyond the max. error.'

2017-10-31 Thread Franco Venturi
A few days ago at work our Kudu servers started having fatal errors and shutting down with the following error message: Couldn't get the current time: Clock unsynchronized. Status: Service unavailable: Error: Clock synchronized but error wastoo high (1016 us). After some research

Re: Change Data Capture (CDC) with Kudu

2017-09-23 Thread Franco Venturi
and hole punching, so you should verify that any tool you use supports that). Disaster Recovery is an oft-requested feature, but one that Kudu developers have been unable to prioritize yet. Would you or your someone on your team be interested in working on this? On Thu, Sep 21, 2017 at 7:12 PM Franco

Change Data Capture (CDC) with Kudu

2017-09-21 Thread Franco Venturi
wondering if there's any way in Kudu to achieve something like that (possibly mining the WALs, since my understanding is that each change gets applied to the WALs first). Thanks, Franco Venturi

Re: Data encryption in Kudu

2017-05-02 Thread Franco Venturi
esponses to some other comments below. On Sat, Apr 29, 2017 at 8:35 PM, Franco Venturi < fvent...@comcast.net > wrote: - also from the security point of view, since the encryption happens at the client side, the data that is transfered on the network between the client and the se

Re: Data encryption in Kudu

2017-04-29 Thread Franco Venturi
In the last couple of days I did some reading on Oracle TDE (Transparent Data Encryption) and had some discussions with people at work, and that helped me clarify my ideas about encryption in Kudu. - The most important thing I realized is that there are basically (at least) two