is it going to make it in 3.0? On Tue, Apr 21, 2020 at 9:24 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> wrote:
> Unfortunately, the short answer is no. Please refer the last part of > discussion on the PR https://github.com/apache/spark/pull/24922 > > Unless we get any native implementation of this, I guess this project is > most widely known implementation for RocksDB backend state store - > https://github.com/chermenin/spark-states > > On Wed, Apr 22, 2020 at 11:32 AM kant kodali <kanth...@gmail.com> wrote: > >> Hi All, >> >> 1. is RosckDB backend available in 3.0 preview? >> 2. if RocksDB can store intermediate results of a stream-stream join can >> I run streaming join queries forever? forever I mean until I run out of >> disk. or put it another can I run the stream-stream join queries for years >> if necessary (imagine I got lot of disk capacity but not a whole lot of >> RAM)? >> 3. Does is to do incremental checkpointing to HDFS? >> >> Thanks! >> >>
