Github user sethah commented on the issue:
https://github.com/apache/spark/pull/17360
@yanboliang Thanks for your feedback! The design of the optimizer
interface, or even whether it should be included at all, is definitely open for
discussion and your suggestions are much appreciated. If SPARK-17136 proceeds
as you suggest (internal optimization API that allows users to register
optimizers) then it is possible that this PR does not conflict with that JIRA
(though I don't know about the details of that, so even that I'm not sure of).
However, that matter is far from settled. If we end up deciding to provide the
external optimizer API as is currently suggested in that JIRA, then these two
_do_ conflict. If we add the ability to specify parameter bounds on the
estimator, then add an optimizer API, we have added yet more optimizer
parameters to the estimator that can conflict with parameters of the optimizer
provided to the estimator.
My point is that I think these are two competing approaches and we should
settle on one over the other before we make API changes that cannot be undone.
I'm open to potentially changing the design of SPARK-17136, but we need to
decide on something first.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
