Here I have added what I had programmed.
With sklearn's LogisticRegression(), how can I see the parameters it has
found after .fit() where the cost is minimal? I use the book of Geron about
scikit-learn and tensorflow and on page 137 he trains the model of petal
widths. I did the following:
iris=datasets.load_iris()
a1=iris['data'][:,3:]
y=(iris['target']==2).astype(int)
log_reg=LogisticRegression()
log_reg.fit(a1,y)
log_reg.coef_
array([[2.61727777]])
log_reg.intercept_
array([-4.2209364])
I did the logistic regression myself with Gradient Descent or
Newton-Raphson as I learned from my Coursera course and respectively from
my book of Bishop. I used the Gradient Descent method like so:
from sklearn import datasets
iris=datasets.load_iris()
a1=iris['data'][:,3:]
A1=np.c_[np.ones((150,1)),a1]
y=(iris['target']==2).astype(int).reshape(-1,1)
lmda=1
from scipy.special import expit
def logreg_gd(w):
z2=A1.dot(w)
a2=expit(z2)
delta2=a2-y
w=w-(lmda/len(a1))*A1.T.dot(delta2)
return w
w=np.array([[0],[0]])
for i in range(0,100000):
w=logreg_gd(w)
In [6219]: w
Out[6219]:
array([[-21.12563996],
[ 12.94750716]])
I used Newton-Raphson like so, see Bishop page 207,
from sklearn import datasets
iris=datasets.load_iris()
a1=iris['data'][:,3:]
A1=np.c_[np.ones(len(a1)),a1]
y=(iris['target']==2).astype(int).reshape(-1,1)
def logreg_nr(w):
z1=A1.dot(w)
y=expit(z1)
R=np.diag((y*(1-y))[:,0])
H=A1.T.dot(R).dot(A1)
tmp=A1.dot(w)-np.linalg.inv(R).dot(y-t)
v=np.linalg.inv(H).dot(A1.T).dot(R).dot(tmp)
return v
w=np.array([[0],[0]])
for i in range(0,10):
w=logreg_nr(w)
In [5149]: w
Out[5149]:
array([[-21.12563996],
[ 12.94750716]])
Notice how much faster Newton-Raphson goes than Gradient Descent. But they
give the same result.
How can I see which parameters LogisticRegression() found? And should I
give LogisticRegression other parameters?
On Sat, Jun 8, 2019 at 11:34 AM Eric J. Van der Velden <
ericjvandervel...@gmail.com> wrote:
> Hello,
>
> I am learning sklearn from my book of Geron. On page 137 he learns the
> model of petal widths.
>
> When I implements logistic regression myself as I learned from my Coursera
> course or from my book of Bishop I find that the following parameters are
> found where the cost function is minimal:
>
> In [6219]: w
> Out[6219]:
> array([[-21.12563996],
> [ 12.94750716]])
>
> I used Gradient Descent and Newton-Raphson, both give the same answer.
>
> My question is: how can I see after fit() which parameters
> LogisticRegression() has found?
>
> One other question also: when I read the documentation page,
> https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression,
> I see a different cost function as I read in the books.
>
> Thanks.
>
>
>
>
_______________________________________________
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn
