DL 公式_Sigmoid_Softmax_Tanh

---

Logistic function

f(x)=Le−k(x−x0)

e = 自然底数 the natural logarithm base (also known as Euler’s number)
x0 = 中点横坐标 the x-value of the sigmoid’s midpoint
L = 最大值 the curve’s maximum value, and
k = 陡峭程度 the steepness of the curve
Reference: wiki

---

Sigmoid function

f(x)=11+e−x

f′(x)=e−x(1+e−x)2=f(x)(1−f(x))
Reference: Logistic Regression

Sigmoid函数是Logistic函数的特殊情况

---

Softmax

σ(z)j=ezj∑Kk=1ezk for j = 1, …, K

Softmax Regression 别名 Multinomial logistic regression。
Softmax function是sigmoid function在多个类的扩展

当K=2时，Softmax function可以推导成sigmoid function:

Pr(Yi=1)=11+e−β⋅Xi

Pr(Yi=0)=1−Pr(Yi=1)=e−β⋅Xi1+e−β0⋅Xi

Pr(Yi=1)=eβ1⋅Xi∑0≤c≤Keβc⋅Xi=eβ1⋅Xieβ0⋅Xi+eβ1⋅Xi=1e(β0−β1)⋅Xi+1=11+e−β⋅Xi

Pr(Yi=0)=eβ0⋅Xi∑0≤c≤Keβc⋅Xi=eβ0⋅Xieβ0⋅Xi+eβ1⋅Xi=e(β0−β1)⋅Xie(β0−β1)⋅Xi+1=e−β⋅Xi1+e−β⋅Xi

其中 β=−(β0−β1)

Reference1: Softmax和Sigmoid的关系

Reference2: What is the intuition behind SoftMax function?

---

tanh (Hyperbolic Tangent)

f(x)=ex−e−xex+e−x=21+e−2x−1 (sigmoid函数在x轴收缩2倍, y轴扩大2倍再减1)

=1−2e−xex+e−x=1−2e2x+1

f2(x)=1−4exe−x(ex+e−x)2=1−4e2x(e2x+1)2

f′(x)=4e2x(e2x+1)2=1−f2(x)

---

Jaccard index

Intersection over Union
Jaccard similarity coefficient
J(A,B)=|A∩B||A∪B|=|A∩B||A|+|B|−|A∩B|

---