Re: Data inconsistency after restart

2017-12-08 Thread David Alves
ible. Can it be that after the scan it's just that enough time has elapsed so that all replicas are caught up? I'd say this is likely the case. Br, Petter 2017-12-07 21:39 GMT+01:00 David Alves <davidral...@gmail.com>: Hi Petter I'd like to clarify what exactly happened and exactly

Re: Data inconsistency after restart

2017-12-07 Thread David Alves
Hi Petter I'd like to clarify what exactly happened and exactly what are you referring to as "inconsistency". From what I understand of the first error you observed, the Kudu was underprovisioned, memory wise, and the ingest jobs/queries failed. Is that right? Since Kudu doesn't have atomic

Re: Data inconsistency after restart

2017-12-07 Thread David Alves
(re-sending as apparently a previous version of this message got lost) Hi Petter I'd like to clarify what exactly happened and exactly what are you referring to as "inconsistency". From what I understand of the first error you observed, the Kudu was underprovisioned, memory wise, and the

Re: kudu's Cluster scale question!

2017-12-04 Thread David Alves
------- > wenye...@163.com -- David Alves

Re: Time-travel reads via SQL query

2017-11-28 Thread David Alves
Hi Mauricio Andrew is right. That feature already exists in some form. With READ_AT_SNAPSHOT you can provide a timestamp which will be the timepoint under which all the scans are performed. Note that, while generally supported and functionally tested, we haven't focused a lot of resources

Re: RPC and Service difference?

2017-04-28 Thread David Alves
Hi The acceptor thread only distributes work, it's very unlikely that is a bottleneck. Same goes for the number of workers, since the number of threads pulling data is defined by impala. What is "extremely" slow in this case? Some things to check: It seems like this is scanning only 5

Re: Some bulk requests are missing when a tserver stopped

2017-04-27 Thread David Alves
cala:214) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool > Executor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo > lExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > 2017-04-26 0:28 GMT+09:00 David Alves <davidral...@gmail

Re: Kudu Table Design Question

2017-04-27 Thread David Alves
The suggestion of 20-30 per tablet is more about the number of available cores than the size of the data. Tools like impala derive parallelism from the number of tablets thus having that count adjusted (but not necessarily equal to) to the core count gives you a good performance tradeoff. Of

Re: Bad insert performance of java kudu-client

2017-04-24 Thread David Alves
cally as well using your code. > > -Todd > > On Mon, Apr 24, 2017 at 10:49 AM, David Alves <davidral...@gmail.com> > wrote: > >> Hi Pavel >> >> Interesting, Thanks for sharing those numbers. >> I assume you weren't using AUTOFLUSH_BACKGROUND for

Re: Question about memory_limit_hard_bytes and

2017-04-24 Thread David Alves
Hi Jason memory_limit_hard_bytes refers to all the memory consumed by the tablet server. It has an effect in many things but arguably the biggest effect is that the tablet server will reject all writes when memory consumption reaches this limit. block_cache_capacity_mb dictates how much

Re: [ANNOUNCE] Two new Kudu committer/PMC members

2016-09-12 Thread David Alves
Congrats Alexey and Will!! On Mon, Sep 12, 2016 at 3:55 PM, Todd Lipcon wrote: > Hi Kudu community, > > It's my great pleasure to announce that the PMC has voted to add both > Alexey Serbin and Will Berkeley as committers and PMC members. > > Alexey has been contributing for a