[深度学习论文笔记][Adversarial Examples] Deep Neural Networks are Easily Fooled: High Confidence Predictions
来源:互联网 发布:淘宝助理怎么导出图片 编辑:程序博客网 时间:2024/05/29 13:42
puter Vision and Pattern Recognition (CVPR). IEEE, 2015. (Citations: 190).
Produce images that are completely unrecognizable to humans, but CNNs believe to be recognizable objects with 99.99% confidence.
2 Implication of Adversarial Examples
For example, one can imagine a security camera that relies on face or voice recognition being compromised. Swapping white-noise for a face, fingerprints, or a voice might beespecially pernicious since other humans nearby might not recognize that someone is attempting to compromise the system.
Another area of concern could be image-based search engine rankings: background patterns that a visitor does not notice could fool a CNN-driven search engine into thinking a
page is about an altogether different topic.
Like [Simonyan et al. 2013], compute adversarial examples by optimization, but we are optimize wrt the posterior probability (softmax output).
Starting with random noise, repeat update until the CNN confidence for the target class reaches 99.99%. Adding regularization makes images more recognizable but results in
slightly lower confidence scores. See the results in Fig.
4 GA Based
4.1 GA Approach
See Fig.
• Organisms: synthetic images that are used to fool CNN.
• fitness function: highest prediction value a CNN makes for that image belonging to a class.
• selection: MAP-Elites: keep the best individual found so far for each objective (each class).
4.2 Direct Encoding
Three integers (H, S, V) for each pixel of the image. Each pixel value is initialized with uniform random noise within [0, 255] range.
Each pixel value has probability p of being mutated by a polynomial mutation operator. p starts with 0.1 and drops by half every 1000 generations.
4.3 Indirect Encoding
Use compositional pattern-producing network (CPPN). CPPN is similar to CNN, which takes in the (x, y) position of a pixel as input, and outputs the tuple of HSV values for
that pixel. CPPN can evolve complex, regular images that resemble natural and man-made objects.
Evolution determines the topology, weights, and activation functions of each CPPN network in the population. Thus, elements in the genome can affect multiple parts of the
iamge. CPPN networks start with no hidden nodes, and nodes are added over time, encouraging evolution to first search for simple, regular images before adding complexity
4.4 Results
See Fig. 10.4. GA exploits specific discriminative features corresponding to each class learned by CNN. This is because evolution need only to produce features that are unique to, or discriminative for, a class, rather than produce an image that contains all of the typical features of a class. Many of the CPPN images feature a pattern repeated many times. The extra copies make the CNN more confident that the image belongs to the target class. It also show that CNN tend to learn low- and middle-level features rather than the global structure of objects.
Larger datasets are a way to make CNN more difficult to fool. Because there are more cat and dog classes, the GA had difficulty finding an image that scores high in a specific
dog category (e.g. Japanese spaniel), but low in any other related categories (e.g. Blenheim spaniel), which is necessary to produce a high confidence given that the final CNN layer is softmax. This explanation suggests that datasets with more classes can help ameliorate fooling.
It is not easy to prevent the CNN from being fooled by retraining them with fooling images (be recognize as a new “fooling images” class). While retrained CNN learn to
classify the negative examples as fooling images, a new batch of fooling images can be produced that fool these new networks, even after many retraining iterations.
5 Analysis
Discriminative models learn p(y|X) directly. They create decision boundaries that partition data into classification regions, see Fig. In a high-dimensional input space, the area a discriminative model allocates to a class may be much larger than the area occupied by training examples for that class. Synthetic images far from the decision boundary and deep into a classification region may produce high confidence predictions even though they are far from the natural images in the class. The large regions of high confidence exist in certain discriminative models due to a combination of their locally linear nature and high-dimensional input space.
In contrast, a generative model that represents the complete joint density p(X, y) = p(y|X)p(X). Such models may be more difficult to fool because fooling images could be
recognized by their low marginal probability p(X), and the CNNs confidence in a label prediction for such images could be discounted when p(X) is low.
6 References
[1]. Evolving AI Lab. https://www.youtube.com/watch?v=M2IebCN9Ht4.
[2]. CVPR 2015. http://techtalks.tv/talks/deep-neural-networks-are-easily-fooled-high-confidence-predictions-for-unrecog61573/.
- [深度学习论文笔记][Adversarial Examples] Deep Neural Networks are Easily Fooled: High Confidence Predictions
- Deep Neural Networks are Easily Fooled:High Confidence Predictions for Unrecognizable Images
- 对抗样本论文学习:Deep Neural Networks are Easily Fooled
- [深度学习论文笔记][Adversarial Examples] Intriguing properties of neural networks
- PS: Deep Neural Networks Are Easily Fooled___CVPR2015
- 【深度学习论文笔记】Deep Neural Networks for Object Detection
- [深度学习论文笔记][Adversarial Examples] Explaining and Harnessing Adversarial Examples
- [深度学习论文笔记][Image Classification] ImageNet Classification with Deep Convolutional Neural Networks
- [深度学习论文笔记][Visualizing] Understanding Neural Networks Through Deep Visualization
- [深度学习论文笔记][Human Pose Estimation] DeepPose: Human Pose Estimation via Deep Neural Networks
- 【深度学习论文笔记:Recognition】:Deep Neural Networks for Object Detection
- Neural Networks and Deep Learning学习笔记ch6 - 深度学习
- [深度学习论文笔记][arxiv 1711] Non-local Neural Networks
- 深度学习论文理解3:Flexible, high performance convolutional neural networks for image classification
- 【深度学习经典论文翻译1】AlexNet-ImageNet Classification with Deep Convolutional Neural Networks全文翻译
- [deep compression] compressing deep neural networks 论文学习
- [深度学习论文笔记][Neural Arts] Inceptionism: Going Deeper into Neural Networks
- [深度学习论文笔记][Recurrent Neural Networks] Visualizing and Understanding Recurrent Networks
- Fragment的添加步骤
- C#获取路径总结
- Spark之Scala学习之路(三)
- 微信xml解析
- Redis系列~主从复制(十八)
- [深度学习论文笔记][Adversarial Examples] Deep Neural Networks are Easily Fooled: High Confidence Predictions
- RabbitMQ-三、Java使用--4、主题(Topic)
- ubuntu 安装atom
- oracle function用法
- Android产品研发(八)-->App数据统计
- 29、SSH框架-Spring+Mybatis+SpringMVC+Maven+MySql搭建实例(4)
- 卷积神经网络在自然语言处理的应用
- 深度学习主机配置 gtx1080 + cuda + tensorflow + ubuntu16
- POJ 2001 Shortest Prefixes