Re: Reference configs for HA / RocksDB / YARN / Zookeeper / HDFS

2017-03-10 Thread Robert Metzger
Hi Dave, Let me answer your questions: 1. The RocksDB state backend always stores the data on local disks for speed. The back up is done to HDFS or any other distributed file system. The local data directory is configured automatically by YARN. 2. You need to manually configure zookeeper in the f

Reference configs for HA / RocksDB / YARN / Zookeeper / HDFS

2017-03-10 Thread Torok, David
Hi, Forgive me if parts of this question have been answered before but I'd like help in resolving some bits of confusion from the documentation and the fact that I haven't been able to find a good example anywhere for an enterprise-style setup. If anyone has a sample HA / Yarn / ZK / RocksDB

Reference configs for HA / RocksDB / YARN / Zookeeper / HDFS

2017-03-10 Thread Torok, David
Hi, Forgive me if parts of this question have been answered before but I'd like help in resolving some bits of confusion from the documentation and the fact that I haven't been able to find a good example anywhere for an enterprise-style setup. If anyone has a sample HA / Yarn / ZK / RocksDB