pytorch loss function.

BCELoss

Creates a criterion that measures the Binary Cross Entropy between the target and the output

$sigmoid(x) = \dfrac{1}{1+e^{(-x)}}$

y 是真实标签，$y\in \{0,1\}$, 那么对于每一个样本的概率为： $P(x_i, y_i)=P(y_i=1|x_i)^{y_i}P(y_i=0|x_i)^{1-y_i}$ $=P(y_i=1|x_i)^{y_i}(1-P(y_i=1|x_i))^{1-y_i}$

$loss(p,t)=−\dfrac{1}{N}\sum_{i=1}^{N}=\dfrac{1}{N}[t_i∗log(p_i)+(1−t_i)∗log(1−p_i)]$

example:

torch.nn.CrossEntropyLoss

This criterion combines nn.LogSoftmax() and nn.NLLLoss() in one single class.

It is useful when training a classification problem with C classes. If provided, the optional argument weight should be a 1D Tensor assigning weight to each of the classes. This is particularly useful when you have an unbalanced training set.

example:

torch.nn.NLLloss

The negative log likelihood loss. It is useful to train a classification problem with C class.

input 是已经通过 log_softmax 层的输入。loss 是对应样本中真实标签对应的值的负数。

NLLloss
$\ell(x, y) = L = \{l_1,\dots,l_N\}^\top, \quad l_n = - w_{y_n} x_{n,y_n}, \quad w_{c} = \text{weight}[c] \cdot \mathbb{1}\{c \not= \text{ignore_index}\}$

example：

MultiMarginLoss

$loss = \dfrac{1}{N}\sum_{j\ne y_i}^{N}max(0,s_j - s_{y_i}+\Delta)$

$s_{yi}$ 表示其真实标签对应的值，那么其他非真实分类的结果凡是大于 $s_{yi}−\Delta$ 这个值的，都对最后的结果 $loss$ 产生影响，比这个值小的就没事～

example:

nn.L1loss

$L1(\hat{y}, y)=\dfrac{1}{m}\sum|\hat{y}_i−y_i|$

nn.MSEloss

$L2(\hat{y}, y)=\dfrac{1}{m}\sum|\hat{y}_i−y_i|^2$