Hello everyone! I have been working on Spark history server that uses MongoDB as a datastore for processed events to iterate on idea that Spree project uses for Spark UI. Project was originally designed to improve on standalone history server with reduced memory footprint.
Project lives here: https://github.com/lightcopy/history-server These are just very early days of the project, sort of pre-alpha (some features are missing, and metrics in some failed jobs cases are questionable). Code is being tested on several 8gb and 2gb logs and aims to lower resource usage since we run history server together with several other systems. Would greatly appreciate any feedback on repository (issues/pull requests/suggestions/etc.). Thanks a lot! Cheers, Ivan
