- 线性回归推导过程我们假设 θ \theta θ 与 b b b 为模型参数，X X X 为输入数据的特征，y y y 为输入数据的目标值（标签），η \eta η 为学习率
∂ L ( θ , b , X , y ) ∂ θ = ∂ ∂ θ ( 1 2 ( f ( X ) − y ) 2 ) = ∂ ∂ θ ( 1 2 ( θ ⋅ X + b − y ) 2 ) = ( θ ⋅ X + b − y ) ⋅ X \frac{\partial L(\theta,b,X,y)}{\partial \theta}=\frac{\partial}{\partial \theta} (\frac{1}{2}(f(X)-y)^2)=\frac{\partial}{\partial \theta} (\frac{1}{2}(\theta \cdot X+b-y)^2)=(\theta \cdot X + b -y)\cdot X ∂θ∂L (θ,b ,X ,y )=∂θ∂(2 1 (f (X )−y )2 )=∂θ∂(2 1 (θ⋅X +b −y )2 )=(θ⋅X +b −y )⋅X
∂ L ( θ , b , X , y ) ∂ b = ∂ ∂ b ( 1 2 ( f ( X ) − y ) 2 ) = ∂ ∂ b ( 1 2 ( θ ⋅ X + b − y ) 2 ) = θ ⋅ X + b − y \frac{\partial L(\theta,b,X,y)}{\partial b}=\frac{\partial}{\partial b} (\frac{1}{2}(f(X)-y)^2)=\frac{\partial}{\partial b} (\frac{1}{2}(\theta \cdot X+b-y)^2)=\theta \cdot X + b -y ∂b ∂L (θ,b ,X ,y )=∂b ∂(2 1 (f (X )−y )2 )=∂b ∂(2 1 (θ⋅X +b −y )2 )=θ⋅X +b −y
θ t + 1 ← θ t − η ⋅ ∂ L ( θ t , b t , X , y ) ∂ θ t = θ t − η ⋅ ( θ t ⋅ X + b t − y ) ⋅ X \theta_{t+1} \leftarrow \theta_{t}-\eta \cdot \frac{\partial L(\theta_t,b_t,X,y)}{\partial \theta_t}=\theta_t-\eta \cdot (\theta_t \cdot X + b_t -y)\cdot X θt +1 ←θt −η⋅∂θt ∂L (θt ,b t ,X ,y )=θt −η⋅(θt ⋅X +b t −y )⋅X b t + 1 ← b t − η ⋅ ∂ L ( θ t , b t , X , y ) ∂ b t = b t − η ⋅ ( θ t ⋅ X + b t − y ) b_{t+1}\leftarrow b_t-\eta \cdot \frac{\partial L(\theta_t,b_t,X,y)}{\partial b_t}=b_t-\eta\cdot(\theta_t \cdot X+b_t-y)b t +1 ←b t −η⋅∂b t ∂L (θt ,b t ,X ,y )=b t −η⋅(θt ⋅X +b t −y )
Original: https://blog.csdn.net/weixin_44944722/article/details/125808852
Author: WBZhang2022
Title: 深度学习小记 – 正则化，优化器，线性回归，逻辑斯蒂回归
原创文章受到原创版权保护。转载请注明出处：https://www.johngo689.com/635821/
转载文章受原作者版权保护。转载请注明原作者出处！