# 【机器学习】算法原理详细推导与实现(二):逻辑回归

## logistic函数

[g(z)=\frac{1}{1+e^{-z}} ]

[h_\theta(x)=g(\theta^Tx)=\frac{1}{1+e^{-\theta^Tx}} ]

(g(z)) 被称为 sigmoid&#x51FD;&#x6570; ，也通常被称为 logistic&#x51FD;&#x6570;，它的函数图像是：

## 逻辑回归

[P(y=1|x;\theta)=h_\theta(x) ]

[P(y=0|x;\theta)=1-h_\theta(x) ]

[P(y|x;\theta)=(h_\theta(x))^y(1-h_\theta(x))^{1-y} ]

[\begin{split} L(\theta)&=P(\vec{y}|X;\theta) \ &=\prod_{i=1}^mP(y^{(i)}|x^{(i)};\theta) \ &=\prod_{i=1}^m(h_\theta(x^{(i)}))^{y^{(i)}}(1-h_\theta(x^{(i)}))^{1-{y^{(i)}}} \end{split} ]

[\begin{split} l(\theta)&=logL(\theta) \ &=\sum_{i=1}^m{y^{(i)}logh(x^{(i)})+(1-y^{(i)})log(1-h(x^{(i)}))} \end{split} ]

[\theta:=\theta-\alpha\frac{\partial J(\theta)}{\partial\theta}=>\theta:=\theta-\alpha\nabla_\theta J(\theta) ]

[\theta:=\theta+\alpha\nabla_\theta l(\theta)=>\theta:=\theta+\alpha\frac{\partial l(\theta)}{\partial\theta} ]

[\begin{split} \frac{\partial l(\theta)}{\partial\theta_j}&=(y\frac{1}{g(\theta^Tx)}-(1-y)\frac{1}{1-g(\theta^Tx)})\frac{\partial}{\partial\theta_j}g(\theta^Tx) \ &=(y\frac{1}{g(\theta^Tx)}-(1-y)\frac{1}{1-g(\theta^Tx)})g(\theta^Tx)(1-g(\theta^Tx))\frac{\partial}{\partial\theta_j}\theta^Tx \ &=(y(1-g(\theta^Tx))-(1-y)g(\theta^Tx))x_j \ &=(y-g(\theta^Tx))x_j \ &=(y-h_{\theta}(x))x_j \end{split} ]

[\frac{\partial l(\theta)}{\partial\theta_j}=\sum_{i=1}^m{(y-h_{\theta}(x))x_j} ]

[\theta_j:=\theta_j+\sum_{i=1}^m{(y^{(i)}-h_{\theta}(x^{(i)}))x^{(i)}_j} ]

## 逻辑回归是分类为什么叫做回归

[z=\theta x+b ]

[y=\frac{1}{1+e^{-(\theta x+b)}} ]

[\ln\frac{y}{1-y}=z=\theta x+b ]

(y)看作样本(x)为正例的可能性，相应的(1-y)就是样本(x)为反例的可能性，两者的比值(\frac{y}{1-y})叫做几率（odds），取对数(\ln\frac{y}{1-y})后叫做对数几率（logistic odds），对数几率与(x)是线性关系，所以可以称作”回归”。

## 鸢尾花分类

# &#x635F;&#x5931;&#x51FD;&#x6570;
def computeCost(theta, X, y):
theta = np.matrix(theta)
X = np.matrix(X)
y = np.matrix(y)
first = np.multiply(-y, np.log(sigmoid(X * theta.T)))
second = np.multiply((1 - y), np.log(1 - sigmoid(X * theta.T)))
return np.sum(first - second) / (len(X))

# &#x68AF;&#x5EA6;&#x4E0B;&#x964D;
def gradient(theta, X, y):
theta = np.matrix(theta)
X = np.matrix(X)
y = np.matrix(y)

parameters = int(theta.ravel().shape[1])
grad = np.zeros(parameters)

error = sigmoid(X * theta.T) - y

for i in range(parameters):
term = np.multiply(error, X[:, i])
grad[i] = np.sum(term) / len(X)

return grad

accuracy = 99%

Original: https://www.cnblogs.com/TTyb/p/10976291.html
Author: TTyb
Title: 【机器学习】算法原理详细推导与实现(二):逻辑回归

(0)