# 深度学习基础（基础知识0）

[En]

In fact, in-depth learning of all kinds of networks is based on the accumulation of all kinds of basic knowledge. Today, I mainly want to explain these three clearly: gradient descent, neural network, back propagation algorithm.

[En]

Neural network (the cornerstone of everything)

[En]

What is a neural network?

## 1、感知器

[En]

Historically, scientists have wanted to simulate the human brain and build machines that can think. Why can people think? Scientists have found that the reason lies in the neural network of the human body.

[En]

To simplify the model, we agree that there are only two possibilities for each input: 1 or 0. If all the inputs are 1, it means that all the conditions are true, the output is 1; if all the inputs are 0, the condition is not true, the output is 0.

## 2、感知器的例子

[En]

Let’s look at an example. The annual game animation exhibition is being held in the city, and Xiao Ming can’t make up his mind whether to visit it on the weekend.

[En]

He decided to consider three factors.

## 3、权重和阈值

[En]

When you see this, you must ask: if some factors are true and other factors are not, what is the output? For example, the weekend is fine and the tickets are not expensive, but Xiao Ming can’t find a companion, so does he still want to visit?

[En]

The above weight indicates that the weather is the decisive factor, while companion and price are secondary factors.

[En]

The above decision-making process is expressed mathematically as follows.

## 4、决策模型

[En]

A single perceptron makes up a simple decision model that can already be used. In the real world, the actual decision-making model is much more complex, which is a multi-layer network composed of multiple perceptrons.

[En]

In the image above, the bottom perceptron receives the external input, makes a judgment, and then sends a signal as the input of the upper perceptron until the final result is obtained. (note: there is still only one output from the perceptron, but it can be sent to multiple targets. )

## 5、矢量化

[En]

In order to facilitate the later discussion, some mathematical processing of the above model is needed.

[En]

The perceptron model becomes like this.

## 6、神经网络的运作过程

[En]

The construction of a neural network needs to meet three conditions.

[En]

In other words, you need to draw the picture above in advance.

[En]

Therefore, the operation of the neural network is as follows.

1. 确定输入和输出
2. 找到一种或多种算法，可以从输入得到输出
3. 找到一组已知答案的数据集，用来训练模型，估算 wb
4. 一旦新的数据产生，输入模型，就可以得到结果，同时对 wb进行校正

## 7、神经网络的例子

[En]

The following is an example of automatic license plate recognition to explain the neural network.

[En]

A set of identified license plate photos are used as training set data to input the model. Constantly adjust various parameters until the combination of parameters with the highest accuracy is found. When you get the new photos, you can give the results directly.

## 8、输出的连续性

[En]

This requires a little simple mathematical modification.

[En]

The original output curve is as follows.

[En]

Now it’s like this.

&#x394;&#x3C3;&#x394;w&#x394;b之间是线性关系，变化率是偏导数。这就有利于精确推算出 wb的值了。

[En]

Why use nonlinear activation functions?

ANN网络：输入层/输入神经元，输出层/输出神经元，隐层/隐层神经元，权值，偏置，激活函数

ANN网络训练过程：

[En]

The gradient descent method is the fastest solution.

[En]

The derivation is based on the premise that we already know the gradient of the current point in advance. But the truth is, we don’t know.

[En]

So how to solve it?

[En]

So the steps are:

step1、前向传播：

1.输入层—->隐含层：

2.隐含层—->输出层：

Step 2 反向传播

1.计算总误差

2.隐含层—->输出层的权值更新：

3.隐含层—->隐含层的权值更新：

1、反向传播的原理

[En]

First, there are two things to keep in mind:

1.反向传播算法告诉我们当我们改变权值（weights）和偏置（biases）的时候，损失函数改变的速度。
2.反向传播也告诉我们如何改变权值和偏置以改变神经网络的整体表现。

2.一个很好的例子

[En]

Suppose you have a network layer like this:

[En]

Now give them an initial value, as shown in the following figure:

w5=0.40,w6=0.45,w7=0.50,w8=0.88

Step 1 前向传播

1.输入层—->隐含层：

2.隐含层—->输出层：

Step 2 反向传播

1.计算总误差

2.隐含层—->输出层的权值更新：

[En]

The following diagram gives a more intuitive view of how the error is backpropagated:

[En]

Now let’s calculate the value of each formula separately:

（这一步实际上就是对sigmoid函数求导，比较简单，可以自己推导一下）

[En]

The last three are multiplied:

[En]

Looking back at the above formula, we find that:

[En]

If the output layer error is negative, it can also be written as:

3.隐含层—->隐含层的权值更新：

[En]

By the same token, calculate:

[En]

The total value of the two is as follows:

[En]

Finally, multiply the three:

3、算法步骤

[En]

Forward transfer the input signal until the output produces an error, and the error information is backpropagated to update the weight matrix.

BP算法是深度学习中的一种训练和学习方法，用于调整深度网络中每个节点的权值，使网络输出的样本标签与实际标签一致。本文将对反向传播算法的实现细节进行总结。

[En]

Back propagation algorithm is a training and learning method in deep learning, which is used to adjust the weight of each node in the deep network, so that the sample label of the network output is consistent with the actual label. This paper will summarize the implementation details of the backpropagation algorithm.

[En]

The backward propagation algorithm is to propagate the error of the output layer backward to adjust the weight. As shown in figure 1, a network with 1 hidden layer has 2 nodes in the input layer, 3 nodes in the hidden layer and 2 nodes in the output layer.

[En]

The implementation details of backward propagation are as follows:

[En]

An example of calculation is given below:

1.计算各节点的值与激活值
a.隐藏层最顶端节点：(0.35×0.1)+（0.9×0.8）=0.755；

b.隐藏层最低端节点：(0.9×0.6)+（0.35×0.4）=0.68；

c.输出层节点：（0.3×0.68）+（0.9×0.6637）=0.80133

2.后向传播更新权重
a.输出误差:δ=(t-o)(1-o)o=(0.5-0.69)(1-0.69)0.69=-0.0406；
b.更新输出层权重：
W 1 +=W 1 +(δ×i n p u t )=0.3+(-0.0406×0.68)=0.272392
W 2 +=W 2 +(δ×i n p u t )=0.9+(-0.0406×0.6637)=0.87305

c.隐藏层误差与权重更新

δ1 =δ×W 1 =-0.0406×0.272392×(1-o)o=-2.406×10 −3
δ2 =δ×W 2 =-0.0406×0.87305×(1-o)o=-7.916×10 −3

W 3 +=0.1 +(−2.406 ×10 −3 ×0.35 )=0.09916
W 4 +=0.8 +(−2.406 ×10 −3 ×0.9 )=0.7978
W 5 +=0.4 +(−7.916 ×10 −3 ×0.35 )=0.3972
W 6 +=0.6 +(−7.916 ×10 −3 ×0.9 )=0.5928

Original: https://www.cnblogs.com/Anita9002/p/9295420.html
Author: Anita-ff
Title: 深度学习基础（基础知识0）

(0)