Re: Edits log apply performance

2015-01-19 Thread Daniel Haviv
Thought so... Thank you for the info and assistance. BR, Daniel > On 19 בינו׳ 2015, at 19:48, Chris Nauroth wrote: > > Hi Daniel, > > Unfortunately, there likely isn't anything you can do to speed this up. The > process of applying edits will be bound by the available read throughput on > t

Re: Edits log apply performance

2015-01-19 Thread Chris Nauroth
Hi Daniel, Unfortunately, there likely isn't anything you can do to speed this up. The process of applying edits will be bound by the available read throughput on the edits stream and the CPU needs for processing each transaction. This is inherently a single-threaded process in the current implem

Edits log apply performance

2015-01-17 Thread Daniel Haviv
Hi, After restarting the namenode we discovered that there was no checkpoint for quite a while. We are waiting for all the changes to be applied to the fsimage, but it seems like it will take hours. Is there something we can do to expedite the process? Increases parallelism? Something at all?