a ;ZanT@sdZddlmZddlZddlZddlmZddlm Z ddl m Z m Z ddl mZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddl m!Z!dgZ"ddZ#ddZ$ddZ%dddddd d!dZ&dS)"zBPartial dependence plots for regression and classification models.)IterableN)sparse) mquantiles) is_classifier is_regressor) cartesian) check_array)check_matplotlib_support)_safe_indexing)_determine_key_type)_get_column_indices)check_is_fitted)Bunch)DecisionTreeRegressor)RandomForestRegressor)NotFittedError)BaseGradientBoosting)BaseHistGradientBoostingpartial_dependencecCst|trt|dkrtdtdd|Ds8td|d|dkrPtd|dkr`td g}t|jdD]}tt ||dd }|jd|kr|}nNt t ||dd |dd }t |d|drtd tj |d|d|d d}| |qrt||fS)aGenerate a grid of points based on the percentiles of X. The grid is a cartesian product between the columns of ``values``. The ith column of ``values`` consists in ``grid_resolution`` equally-spaced points between the percentiles of the jth column of X. If ``grid_resolution`` is bigger than the number of unique values in the jth column of X, then those unique values will be used instead. Parameters ---------- X : ndarray, shape (n_samples, n_target_features) The data. percentiles : tuple of floats The percentiles which are used to construct the extreme values of the grid. Must be in [0, 1]. grid_resolution : int The number of equally spaced points to be placed on the grid for each feature. Returns ------- grid : ndarray, shape (n_points, n_target_features) A value for each feature at each point in the grid. ``n_points`` is always ``<= grid_resolution ** X.shape[1]``. values : list of 1d ndarrays The values with which the grid has been created. The size of each array ``values[j]`` is either ``grid_resolution``, or the number of unique values in ``X[:, j]``, whichever is smaller. rz/'percentiles' must be a sequence of 2 elements.css&|]}d|kodknVqdS)rN).0xrrElib/python3.9/site-packages/sklearn/inspection/_partial_dependence.py Iz_grid_from_X..z''percentiles' values must be in [0, 1].rrz9percentiles[0] must be strictly less than percentiles[1].z2'grid_resolution' must be strictly greater than 1.axis)Zprobrztpercentiles are too close to each other, unable to build the grid. Please choose percentiles that are further apart.T)ZnumZendpoint) isinstancerlen ValueErrorallrangeshapenpuniquer rZallcloseZlinspaceappendr)X percentilesgrid_resolutionvaluesZfeatureZuniquesrZemp_percentilesrrr _grid_from_X&s8! r,cCs&|||}|jdkr"|dd}|S)Nr)Z%_compute_partial_dependence_recursionndimreshape)estgridfeaturesaveraged_predictionsrrr_partial_dependence_recursionms   r4c Cs g}g}t|r|j}nnt|dd}t|dd} |dkrB|p>| }n|dkrN|n| }|dur|dkrltdn|dkr~tdntd|D]} |} t|D]>\} } t| dr| | | jdd| f<q| | | dd| f<qz*|| }|||t j |dd Wqt y8}ztd |WYd}~qd}~00q|j d}t |j}t|rv|jd krv||d }n.t|r|j dd kr|d }||d }t |j}t|r|jd kr|d d }n.t|r|j dd kr|d }|d d }||fS)N predict_probadecision_functionautozCThe estimator has no predict_proba and no decision_function method.z*The estimator has no predict_proba method.z.The estimator has no decision_function method.ilocrrz0'estimator' parameter must be a fitted estimatorrr-r)rZpredictgetattrr!copy enumeratehasattrr8r'r%Zmeanrr$ZarrayTr.r/r)r0r1r2r(response_method predictionsr3Zprediction_methodr5r6Z new_valuesZX_evaliZvariableZpredeZ n_samplesrrr_partial_dependence_brutews\      "     rBr7)g?gffffff?dlegacy)r>r)r*methodkindcCst|t|s t|s tdt|rBt|jdtjrBtdt|dsdt |sdt |dt d}d}||vrtd |d |t|r|d krtd d } || vrtd |d | |dkr|dkr|dkrtdd}|d kr(t|tr |jdur d}nt|tttfr$d}nd}|dkrt|ttttfs^d} td d | |d krld}|dkrtd |t|dddkrtt|drtd |jddtjt||tjdd} tt|| dd||\} } |dkr`. .. warning:: For :class:`~sklearn.ensemble.GradientBoostingClassifier` and :class:`~sklearn.ensemble.GradientBoostingRegressor`, the `'recursion'` method (used by default) will not account for the `init` predictor of the boosting process. In practice, this will produce the same values as `'brute'` up to a constant offset in the target response, provided that `init` is a constant estimator (which is the default). However, if `init` is not a constant estimator, the partial dependence values are incorrect for `'recursion'` because the offset will be sample-dependent. It is preferable to use the `'brute'` method. Note that this only applies to :class:`~sklearn.ensemble.GradientBoostingClassifier` and :class:`~sklearn.ensemble.GradientBoostingRegressor`, not to :class:`~sklearn.ensemble.HistGradientBoostingClassifier` and :class:`~sklearn.ensemble.HistGradientBoostingRegressor`. Parameters ---------- estimator : BaseEstimator A fitted estimator object implementing :term:`predict`, :term:`predict_proba`, or :term:`decision_function`. Multioutput-multiclass classifiers are not supported. X : {array-like or dataframe} of shape (n_samples, n_features) ``X`` is used to generate a grid of values for the target ``features`` (where the partial dependence will be evaluated), and also to generate values for the complement features when the `method` is 'brute'. features : array-like of {int, str} The feature (e.g. `[0]`) or pair of interacting features (e.g. `[(0, 1)]`) for which the partial dependency should be computed. response_method : {'auto', 'predict_proba', 'decision_function'}, default='auto' Specifies whether to use :term:`predict_proba` or :term:`decision_function` as the target response. For regressors this parameter is ignored and the response is always the output of :term:`predict`. By default, :term:`predict_proba` is tried first and we revert to :term:`decision_function` if it doesn't exist. If ``method`` is 'recursion', the response is always the output of :term:`decision_function`. percentiles : tuple of float, default=(0.05, 0.95) The lower and upper percentile used to create the extreme values for the grid. Must be in [0, 1]. grid_resolution : int, default=100 The number of equally spaced points on the grid, for each target feature. method : {'auto', 'recursion', 'brute'}, default='auto' The method used to calculate the averaged predictions: - `'recursion'` is only supported for some tree-based estimators (namely :class:`~sklearn.ensemble.GradientBoostingClassifier`, :class:`~sklearn.ensemble.GradientBoostingRegressor`, :class:`~sklearn.ensemble.HistGradientBoostingClassifier`, :class:`~sklearn.ensemble.HistGradientBoostingRegressor`, :class:`~sklearn.tree.DecisionTreeRegressor`, :class:`~sklearn.ensemble.RandomForestRegressor`, ) when `kind='average'`. This is more efficient in terms of speed. With this method, the target response of a classifier is always the decision function, not the predicted probabilities. Since the `'recursion'` method implicitly computes the average of the Individual Conditional Expectation (ICE) by design, it is not compatible with ICE and thus `kind` must be `'average'`. - `'brute'` is supported for any estimator, but is more computationally intensive. - `'auto'`: the `'recursion'` is used for estimators that support it, and `'brute'` is used otherwise. Please see :ref:`this note ` for differences between the `'brute'` and `'recursion'` method. kind : {'legacy', 'average', 'individual', 'both'}, default='legacy' Whether to return the partial dependence averaged across all the samples in the dataset or one line per sample or both. See Returns below. Note that the fast `method='recursion'` option is only available for `kind='average'`. Plotting individual dependencies requires using the slower `method='brute'` option. .. versionadded:: 0.24 .. deprecated:: 0.24 `kind='legacy'` is deprecated and will be removed in version 1.1. `kind='average'` will be the new default. It is intended to migrate from the ndarray output to :class:`~sklearn.utils.Bunch` output. Returns ------- predictions : ndarray or :class:`~sklearn.utils.Bunch` - if `kind='legacy'`, return value is ndarray of shape (n_outputs, len(values[0]), len(values[1]), ...) The predictions for all the points in the grid, averaged over all samples in X (or over the training data if ``method`` is 'recursion'). - if `kind='individual'`, `'average'` or `'both'`, return value is :class:`~sklearn.utils.Bunch` Dictionary-like object, with the following attributes. individual : ndarray of shape (n_outputs, n_instances, len(values[0]), len(values[1]), ...) The predictions for all the points in the grid for all samples in X. This is also known as Individual Conditional Expectation (ICE) average : ndarray of shape (n_outputs, len(values[0]), len(values[1]), ...) The predictions for all the points in the grid, averaged over all samples in X (or over the training data if ``method`` is 'recursion'). Only available when kind='both'. values : seq of 1d ndarrays The values with which the grid has been created. The generated grid is a cartesian product of the arrays in ``values``. ``len(values) == len(features)``. The size of each array ``values[j]`` is either ``grid_resolution``, or the number of unique values in ``X[:, j]``, whichever is smaller. ``n_outputs`` corresponds to the number of classes in a multi-class setting, or to the number of tasks for multi-output regression. For classical regression and binary classification ``n_outputs==1``. ``n_values_feature_j`` corresponds to the size ``values[j]``. values : seq of 1d ndarrays The values with which the grid has been created. The generated grid is a cartesian product of the arrays in ``values``. ``len(values) == len(features)``. The size of each array ``values[j]`` is either ``grid_resolution``, or the number of unique values in ``X[:, j]``, whichever is smaller. Only available when `kind="legacy"`. See Also -------- PartialDependenceDisplay.from_estimator : Plot Partial Dependence. PartialDependenceDisplay : Partial Dependence visualization. Examples -------- >>> X = [[0, 0, 2], [1, 0, 0]] >>> y = [0, 1] >>> from sklearn.ensemble import GradientBoostingClassifier >>> gb = GradientBoostingClassifier(random_state=0).fit(X, y) >>> partial_dependence(gb, features=[0], X=X, percentiles=(0, 1), ... grid_resolution=2) # doctest: +SKIP (array([[-4.52..., 4.52...]]), [array([ 0., 1.])]) z5'estimator' must be a fitted regressor or classifier.rz3Multiclass-multioutput estimators are not supportedZ __array__z allow-nan)Zforce_all_finitedtype)r7r5r6zEresponse_method {} is invalid. Accepted response_method names are {}.z, r7zKThe response_method parameter is ignored for regressors and must be 'auto'.)brute recursionr7z3method {} is invalid. Accepted method names are {}.averagerDrIzCThe 'recursion' method only applies when 'kind' is set to 'average'rHN)ZGradientBoostingClassifierZGradientBoostingRegressorZHistGradientBoostingClassifierHistGradientBoostingRegressorrKrrz[Only the following estimators support the 'recursion' method: {}. Try using method='brute'.r6zUWith the 'recursion' method, the response_method must be 'decision_function'. Got {}.F)Z accept_sliceintzall features must be in [0, {}]rC)rGorderrr-cSsg|]}|jdqSrr$rvalrrr rz&partial_dependence..cSsg|]}|jdqSrOrPrQrrrrSrzA Bunch will be returned in place of 'predictions' from version 1.1 (renaming of 0.26) with partial dependence results accessible via the 'average' key. In the meantime, pass kind='average' to get the future behaviour.)rJr+ individual)rTr+)rJrTr+)%rrrr!rZclasses_r%Zndarrayr<rZissparser objectformatjoinrinitrrrr anyZlessr$Zasarrayr Zint32Zravelr,r rBr/r4warningswarn FutureWarningr)Z estimatorr(r2r>r)r*rErFZaccepted_responsesZaccepted_methodsZsupported_classes_recursionZfeatures_indicesr1r+r3r?rrrrs2                   )'__doc__collections.abcrrZZnumpyr%ZscipyrZscipy.stats.mstatsrbaserrZ utils.extmathrZutilsr r r r r Zutils.validationrrZtreerZensembler exceptionsrZ ensemble._gbrZ2ensemble._hist_gradient_boosting.gradient_boostingr__all__r,r4rBrrrrrs<                G ^