Hi, I have no work at the moment so I was wondering if anybody would be interested in me contributing code that generates an Array[ParamMap] for random hyperparameters?
Apparently, this technique can find a hyperparameter in the top 5% of parameter space in fewer than 60 iterations with 95% confidence [1]. I notice that the Spark code base has only the brute force ParamGridBuilder unless I am missing something. Hyperparameter optimization is an area of interest to me but I don't want to re-invent the wheel. So, if this work is already underway or there are libraries out there to do it please let me know and I'll shut up :) Regards, Phillip [1] https://www.oreilly.com/library/view/evaluating-machine-learning/9781492048756/ch04.html
