Thanks! On Fri 26 Oct, 2018, 3:31 PM Andrey Zagrebin, <and...@data-artisans.com> wrote:
> Hi Taher, > > TMs keep state locally while running, in this case RocksDB files already > belong to TM. > You can point it to the same NVME disk location on each node, relevant > Flink options here are: > - io.tmp.dirs > - taskmanager.state.local.root-dirs > This data is transient and has temporary nature. It does not survive a job > failure. > > The checkpoint is a logical snapshot of the operator state for all > involved TMs, > so it belongs to the job and usually uploaded to a distributed file system > available on all TMs. > The location is set in Flink option ‘state.checkpoints.dir'. > This way job can restore from it with different set of TMs. > > Best, > Andrey > > > On 26 Oct 2018, at 08:29, Taher Koitawala <taher.koitaw...@gslab.com> > wrote: > > > > Hi All, > > Our current cluster configuration uses one HDD which is mainly > for root and an other NVME disk per node, [1]we want make sure all TMs > write their own RocksDB files to the NVME disk only, how do we do that? > > > > [2] Is it also possible to specify multiple directories per TMs so that > we have an even spread when the RocksDB files are written? > > > > Thanks, > > Taher Koitawala > >
