See SPARK-18085. That has much of the same goals re: SHS resource usage, and also provides a (currently non-public) API where you could just create a MongoDB implementation if you want.
On Tue, Jul 18, 2017 at 12:56 AM, Ivan Sadikov <ivan.sadi...@gmail.com> wrote: > Hello everyone! > > I have been working on Spark history server that uses MongoDB as a datastore > for processed events to iterate on idea that Spree project uses for Spark > UI. Project was originally designed to improve on standalone history server > with reduced memory footprint. > > Project lives here: https://github.com/lightcopy/history-server > > These are just very early days of the project, sort of pre-alpha (some > features are missing, and metrics in some failed jobs cases are > questionable). Code is being tested on several 8gb and 2gb logs and aims to > lower resource usage since we run history server together with several other > systems. > > Would greatly appreciate any feedback on repository (issues/pull > requests/suggestions/etc.). Thanks a lot! > > > Cheers, > > Ivan > -- Marcelo --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
