""" Representation of a Gaussian mixture model probability distribution. This class allows to estimate the parameters of a Gaussian mixture distribution. 对混合的高斯分布进行参数估计～ """
reg_covar=1e-6： Non-negative regularization added to the diagonal of
covariance.Allows to assure that the covariance matrices are all positive.
非负正则化添加到协方差矩阵对角线上，保证协方差矩阵都是正定的。
max_iter=100: em算法的最大迭代次数
n_init: int, defaults to 1.初始化的次数
init_params: {‘kmeans’, ‘random’}, defaults to ‘kmeans’.
The method used to initialize the weights, the means and the precisionsself.
Must be one of::
- 'kmeans' : responsibilities are initialized using kmeans.
- 'random' : responsibilities are initialized randomly.
- 这里对应的初始化，是指的隐藏变量z的分类所占比例，也就是weight_init，kmeans表示“hard”guess， {0, 1} or {1, . . . , k})
random应该就是”soft”guess吧。
weights_init : shape (n_components, ), optional The user-provided initial weights, defaults to None. If it None, weights are initialized using the init_params method.
means_init : array-like, shape (n_components, n_features), optional. The user-provided initial means, defaults to None, If it None, means are initialized using the init_params method.混合高斯分布的均值初始化，注意shape=(n_components, n_features),有n_components这样的多维高斯分布，每个高斯分布有n_features维度
precisions_init : The user-provided initial precisions (inverse of the covariance matrices), defaults to None. If it None, precisions are initialized using the ‘init_params’ method.The shape depends on ‘covariance_type’::
random_state : int, RandomState instance or None, optional (default=None) 随机数生成器
warm_start : bool, default to False.If ‘warm_start’ is True, the solution of the last fitting is used as initialization for the next call of fit(). This can speed up convergence when fit is called several times on similar problems. 若为True，则fit（）调用会以上一次fit（）的结果作为初始化参数，适合相同问题多次fit的情况，能加速收敛，默认为False。
verbose : int, default to 0. Enable verbose output. If 1 then it prints the current initialization and each iteration step. If greater than 1 then it prints also the log probability and the time needed for each step. 使能迭代信息显示，默认为0，可以为1或者大于1（显示的信息不同）
"""Estimate log probabilities and responsibilities for each sample. Compute the log probabilities, weighted log probabilities per component and responsibilities for each sample in X with respect to the current state of the model. Parameters ---------- X : array-like, shape (n_samples, n_features) Returns ------- log_prob_norm : array, shape (n_samples,) log p(X) log_responsibilities : array, shape (n_samples, n_components) logarithm of the responsibilities """
"""M step. Parameters ---------- X : array-like, shape (n_samples, n_features) log_resp : array-like, shape (n_samples, n_components) Logarithm of the posterior probabilities (or responsibilities) of the point of each sample in X. """
"""Estimate model parameters with the EM algorithm. The method fit the model `n_init` times and set the parameters with which the model has the largest likelihood or lower bound. Within each trial, the method iterates between E-step and M-step for `max_iter` times until the change of likelihood or lower bound is less than `tol`, otherwise, a `ConvergenceWarning` is raised. 迭代终止条件： 迭代次数ｎ_init，极大似然函数或下界函数的增益小于`tol` Parameters ---------- X : array-like, shape (n_samples, n_features) Returns ------- self """
X = _check_X(X, self.n_components)
self._check_initial_parameters(X)
# if we enable warm_start, we will have a unique initialisation
"""Compute the Cholesky decomposition of the precisions. Parameters ---------- covariances : array-like The covariance matrix of the current components. The shape depends of the covariance_type. covariance_type : {'full', 'tied', 'diag', 'spherical'} The type of precision matrices. Returns ------- precisions_cholesky : array-like The cholesky decomposition of sample precisions of the current components. The shape depends of the covariance_type. """
"""Compute the log-det of the cholesky decomposition of matrices. Parameters ---------- matrix_chol : 协方差矩阵的cholesky分解 covariance_type : {'full', 'tied', 'diag', 'spherical'} n_features : int Number of features. Returns ------- log_det_precision_chol : array-like, shape (n_components,) The determinant of the precision matrix for each component. """
Although GMM are often used for clustering, we can compare the obtained clusters with the actual classes from the dataset. We initialize the means of the Gaussians with the means of the classes from the training set to make this comparison valid.
We plot predicted labels on both training and held out test data using a variety of GMM covariance types on the iris dataset. We compare GMMs with spherical, diagonal, full, and tied covariance matrices in increasing order of performance. Although one would expect full covariance to perform best in general, it is prone to overfitting on small datasets and does not generalize well to held out test data.
On the plots, train data is shown as dots, while test data is shown as crosses. The iris dataset is four-dimensional. Only the first two dimensions are shown here, and thus some points are separated in other dimensions.
