Single Shot MultiBox Detector（MXNet）源码阅读笔记（2）

初始化的方法Xavier

We initialize the parameters for all the newly added convolutional layers with the “xavier” method.

论文中所述，我们对新增加的卷积层使用“xavier”的方法进行初始化。
Xavier Initialization很简单，其实就是

Var(Wi)=2nin+nout

nin-输入神经元的数量
nout-输出神经元的数量
则

scale=magnitudeVar(Wi)−−−−−−−−−−√

具体解释见andy’sblog
原理出处见论文Understanding the difficulty of training deep feedforward neural networks
MXnet中代码initializer.py部分

class Xavier(Initializer):    """Initialize the weight with Xavier or similar initialization scheme.    Parameters    ----------    rnd_type: str, optional        Use ```gaussian``` or ```uniform``` to init    factor_type: str, optional        Use ```avg```, ```in```, or ```out``` to init    magnitude: float, optional        scale of random number range    """    def __init__(self, rnd_type="uniform", factor_type="avg", magnitude=3):        super(Xavier, self).__init__(rnd_type=rnd_type, factor_type=factor_type,magnitude=magnitude)        self.rnd_type = rnd_type        self.factor_type = factor_type        self.magnitude = float(magnitude)    def _init_weight(self, _, arr):        shape = arr.shape        hw_scale = 1.        if len(shape) > 2:            hw_scale = np.prod(shape[2:])        fan_in, fan_out = shape[1] * hw_scale, shape[0] * hw_scale        factor = 1.        if self.factor_type == "avg":            factor = (fan_in + fan_out) / 2.0            # 计算方差        elif self.factor_type == "in":            factor = fan_in        elif self.factor_type == "out":            factor = fan_out        else:            raise ValueError("Incorrect factor type")        scale = np.sqrt(self.magnitude / factor)        # 计算范围        if self.rnd_type == "uniform":            random.uniform(-scale, scale, out=arr)        elif self.rnd_type == "gaussian":            random.normal(0, scale, out=arr)        else:            raise ValueError("Unknown random type")