对抗样本生成方法论文阅读笔记

The paper “countermeasure sample generation method for black box intelligent voice software”

For example, in the original picture of a snowy mountain, after adding disturbance, our human eyes still look like a snowy mountain, but in the model, it will be mistakenly identified as a dog.

Because of the existence of antagonistic samples, the security of various fields used in deep learning is difficult to be guaranteed. For example, in the field of autopilot, the on-board speech recognition system is attacked by minor disturbances during input, and the speech recognition system will mistakenly recognize the instructions of passengers, which brings serious security risks to the autopilot system.

In the process of constructing confrontation samples, whether image recognition system or speech recognition system, according to how much information the attacker has mastered the machine learning model, it can be divided into white-box attack and black-box attack.

White-box attacks: attackers can learn about the algorithms used in machine learning and the parameters used by the algorithms. Attackers can interact with machine learning systems in the process of generating adversarial attack data.

Black box attack: the attacker does not know the algorithms and parameters used in machine learning, but the attacker can still interact with the machine learning system, such as passing in any input to observe the output and judge the output.

Aimless attack: the output of the attacked model is fine as long as it is incorrect. If the original image is a kitten, add interference to form a countermeasure sample input into the model, the model output error, the output result can be a puppy, can be a lamb or other, only the requirement is wrong.

In practical applications, it is usually difficult for attackers to obtain the structural information of the recognition model, and the purpose of the attack is to achieve a specific target (for example, to make a certain section of speech recognition into specific instructions). Therefore, the black box target attack is more aggressive and hidden in practical applications.

In order to realize the black box target attack for speech recognition system, this paper proposes a target countermeasure sample generation method for black box intelligent speech software, that is, firefly-gradient countermeasure sample generation method.

The limitations of the previous countermeasure sample generation methods for black-box speech recognition systems:

The target countermeasure sample generation method-firefly-gradient countermeasure sample generation method for black box intelligent voice software proposed in this paper, the contribution points are as follows:

The experiment uses three indicators: speech similarity, the time required to generate countermeasure samples and the success rate of generating antagonistic samples to measure the effectiveness of the method.

Firefly algorithm: in a glowing population, each firefly moves toward the position of the firefly that is brighter than it is. At the same time, there is a certain degree of attraction between fireflies with different brightness. After moving many times, other fireflies gather around the position of the brightest firefly. Belongs to the swarm intelligence algorithm, that is, through different methods to construct a population with multiple individuals, and then constantly find the best individual in the population.

Gradient evaluation method: it is a countermeasure sample generation method for black box intelligent software. The main function of the gradient evaluation formula is to generate the gradient, make a difference between it and the original gradient function value, make minor changes to the original gradient function value, and optimize it, so as to generate more accurate confrontation samples.

Using the firefly algorithm or gradient evaluation method to continue the optimization, if the editing distance between the text content of the generated antagonistic sample and the target text is 0, it means that the target antagonistic sample is generated successfully. The selection of the sample optimization algorithm depends on the editing distance between the text content of the antagonistic sample and the target text: the firefly algorithm is used to find the current optimal individual in a large range, and the gradient evaluation method carries on the local key disturbance and iterates constantly to generate the target countermeasure sample.

The experiment uses three indicators: speech similarity, the time required to generate countermeasure samples and the success rate of generating antagonistic samples to measure the effectiveness of the method.

This shows that the confrontation sample generated by the proposed method can make people can not detect the obvious difference from the original sample. However, there is still a small part of the noise against the sample, which needs to be reduced in the later work.

When the speech similarity and countermeasure sample generation time are slightly worse than the comparison method, the success rate of target countermeasure sample generation is better than that of the comparison method in different types of data sets.

The success rate has been improved, but there is still a lot of room for improvement; the speech similarity between the original audio and the generated anti-audio is relatively low.