# pytorch-损失函数

pytorch loss function.

## Cross Entropy

$$-\log P(X)$$

$$H(p) = -\sum_{i=1}^np_i\log p_i$$

$$H(p,q) = -\sum_{i=1}^np_i\log q_i$$

## BCELoss

Creates a criterion that measures the Binary Cross Entropy between the target and the output

$$sigmoid(x) = \dfrac{1}{1+e^{(-x)}}$$

y 是真实标签，$y\in {0,1}$, 那么对于每一个样本的概率为：

$$P(x_i, y_i)=P(y_i=1|x_i)^{y_i}P(y_i=0|x_i)^{1-y_i}$$

$$=P(y_i=1|x_i)^{y_i}(1-P(y_i=1|x_i))^{1-y_i}$$

$$-y_iP(y_i=1|x_i)-(1-y_i)(1-P(y_i=1|x_i))$$

$$loss(p,t)=−\dfrac{1}{N}\sum_{i=1}^{N}=\dfrac{1}{N}[t_i∗log(p_i)+(1−t_i)∗log(1−p_i)]$$

example:

## torch.nn.CrossEntropyLoss

This criterion combines nn.LogSoftmax() and nn.NLLLoss() in one single class.

It is useful when training a classification problem with C classes. If provided, the optional argument weight should be a 1D Tensor assigning weight to each of the classes. This is particularly useful when you have an unbalanced training set.

example:

## torch.nn.NLLloss

The negative log likelihood loss. It is useful to train a classification problem with C class.

input 是已经通过 log_softmax 层的输入。loss 是对应样本中真实标签对应的值的负数。

NLLloss

$$\ell(x, y) = L = {l_1,\dots,l_N}^\top, \quad l_n = - w_{y_n} x_{n,y_n}, \quad w_{c} = \text{weight}[c] \cdot \mathbb{1}{c \not= \text{ignore_index}}$$

example：

## MultiMarginLoss

$loss = \dfrac{1}{N}\sum_{j\ne y_i}^{N}max(0,s_j - s_{y_i}+\Delta)$

$s_{yi}$ 表示其真实标签对应的值，那么其他非真实分类的结果凡是大于 $s_{yi}−\Delta$ 这个值的，都对最后的结果 $loss$ 产生影响，比这个值小的就没事～

example:

## nn.L1loss

$$L1(\hat{y}, y)=\dfrac{1}{m}\sum|\hat{y}_i−y_i|$$

## nn.MSEloss

$$L2(\hat{y}, y)=\dfrac{1}{m}\sum|\hat{y}_i−y_i|^2$$

Xie Pan

2018-12-07

2021-06-29