Re: Spark history server running on Mongo

2017-07-19 Thread Ivan Sadikov
UI, but still have fair amount of them to add. It is all mentioned in repo readme, by the way. Thanks for clearing things out. Have a good day! Ivan On Thu, 20 Jul 2017 at 5:52 AM, Marcelo Vanzin wrote: > On Tue, Jul 18, 2017 at 7:21 PM, Ivan Sadikov > wrote: > > Repository that

Re: Spark history server running on Mongo

2017-07-18 Thread Ivan Sadikov
distribution, which is preferable in my case. Kind regards, Ivan On Wed, 19 Jul 2017 at 4:44 AM, Ivan Sadikov wrote: > Thanks for JIRA ticket reference! Frankly, I was aware of this work, but > didn't know that there was an API for storage implementation. > > Will try exploring tha

Re: Spark history server running on Mongo

2017-07-18 Thread Ivan Sadikov
re: SHS resource > usage, and also provides a (currently non-public) API where you could > just create a MongoDB implementation if you want. > > On Tue, Jul 18, 2017 at 12:56 AM, Ivan Sadikov > wrote: > > Hello everyone! > > > > I have been working on Spark history

Spark history server running on Mongo

2017-07-18 Thread Ivan Sadikov
Hello everyone! I have been working on Spark history server that uses MongoDB as a datastore for processed events to iterate on idea that Spree project uses for Spark UI. Project was originally designed to improve on standalone history server with reduced memory footprint. Project lives here: htt