Remember that topic partitions will not automatically move between brokers or storage locations, so any topics that became under-replicated when the node went down won't heal themselves. When the disk is replaced, you'll be able to start the broker, and then it should begin doing log recovery and catching up on its replication.
On Thu, May 10, 2018 at 12:06 PM, <andrianjar...@gmail.com> wrote: > > Thanks a lot for the explanation Peter, sounds like what I thought. > > I am just not sure I got the last part. So if a disk on such a broker > fails, and we have kafka version <1, the whole broker dies ? > > What happens when the disk is replaced then ? > > > On May 10, 2018, at 20:42, Peter Bukowinski <pmb...@gmail.com> wrote: > > > > Oops, sorry about the name misspelling, Andrian. (spell-check just tried > to > > correct it again). > > > >> On Thu, May 10, 2018 at 11:41 AM, Peter Bukowinski <pmb...@gmail.com> > wrote: > >> > >> Adrian, > >> > >> Replicas are *always* assigned to different brokers. You cannot, for > >> example, deploy a single broker with a replication factor of 2 or 3 > (with > >> min.insync.replicas of 2 or 3, respectively), even with multiple data > >> directories. > >> > >> At the cluster level, kafka is not aware of an individual broker's > storage > >> topology (single or multiple storage locations). Topic partitions on a > >> single, multi-data directory broker are distributed among storage > locations > >> in a round-robin manner. > >> > >> In a disk failure scenario, you will only lose one replica of all the > >> topic partitions that existed on that disk, assuming you're running > 1.0+. > >> If you're not running 1.0+, then a single disk failure on a broker > >> configured with JBOD will bring down the broker. > >> > >> Hope this helps, > >> > >> Peter Bukowinski > >> > >> On Thu, May 10, 2018 at 1:49 AM, Andrian Jardan < > andrianjar...@gmail.com> > >> wrote: > >> > >>> Hello everyone, > >>> > >>> I was wondering how data is spread across disks when more than 1 data > >>> folder is specified on a broker ? > >>> > >>> I am specifically interested to understand if failure of 3 disks may > lead > >>> to data loss (with replication factor at 3)? > >>> > >>> Or is the data replicated so it resides on 3 brokers, and not 3 > different > >>> data folders ? > >>> > >>> Thanks ! > >>> > >>> — > >>> Andrian Jardan > >>> Infrastructure and DevOps expert > >>> cell: +49 174 2815994 > >>> Skype: macrosdnb > >>> > >>> > >> >
