Optimization inequalities cheatsheet
来源:互联网 发布:树莓派3安装ubuntu 编辑:程序博客网 时间:2024/06/05 23:18
This article is from http://fa.bianp.net/blog/2017/optimization-inequalities-cheatsheet/, just record it.
Most proofs in optimization consist in using inequalities for a particular function class in some creative way. This is a cheatsheet with inequalities that I use most often. It considers class of functions that are convex, strongly convex and
Setting.
∥∇f(y)−∇f(x)∥≤∥x−y∥ |f(x)−f(y)−⟨∇f(x),y−x⟩|≤L2∥y−x∥2 ∥∇2f(x)∥≤L (assuming f is twice differentiable)
f(x)≤f(y)+⟨∇f(x),x−y⟩ 0≤⟨∇f(x)−∇f(y),x−y⟩ f(EX)≤E[f(X)] whereX is a random variable (Jensen’s inequality).
1L∥∇f(x)−∇f(y)∥2≤⟨∇f(x)−∇f(y),x−y⟩ 0≤f(y)−f(x)−⟨∇f(x),y−x⟩≤L2∥x−y∥2 f(x)≤f(y)+⟨∇f(x),x−y⟩−12L∥∇f(x)−∇f(y)∥2
f(x)≤f(y)+⟨∇f(x),x−y⟩−μ2∥x−y∥2 μ2∥x−y∥2≤⟨∇f(x)−∇f(y),x−y⟩ μ2∥x−x∗∥2≤f(x∗)−f(x)
μLμ+L∥x−y∥2+1μ+L∥∇f(x)−∇f(y)∥2≤⟨∇f(x)−∇f(y),x−y⟩
Most of these inequalities appear in the Book: “Introductory lectures on convex optimization: A basic course” by Nesterov, Yurii (2013, Springer Science & Business Media). Another good source (and freely available for download) is the book “Convex Optimization” by
Stephen Boyd and Lieven Vandenberghe.
- Optimization inequalities cheatsheet
- Inequalities
- optimization
- optimization
- Optimization
- optimization
- optimization
- Inconsistent equations and inequalities
- Several Inequalities- Probability
- Inequalities - Elementary mean values
- Inequalities - Holder's inequality
- Inequalities - Minkowski's inequality
- Inequalities - Tchebychef's inequality
- Python CheatSheet
- Exim Cheatsheet
- netcat cheatsheet
- Markdown Cheatsheet
- Laravel cheatsheet
- LeetCode 120. Triangle
- 浅谈 SOAP Webserver 与 Restful Webserver 区别
- Socket学习 - php+Socket多进程处理速学:防止子进程无限增加
- 我的第一篇博客
- 【Apache Hadoop】MapReuce 编程总结-多MapReduce执行
- Optimization inequalities cheatsheet
- Linux 精通Linux的“kill”命令
- Linux 文件编程—fopen函数
- E:before E:after 插入图像
- 如何有效管理自己的时间
- 从决策树到随机森林--小结
- LeetCode 258. Add Digits
- nsSkinEngine制作Windows安装包
- leetcode 11 - Container With Most Water