a ;Za%@sddlZddlZddlmZddlZddlmZmZddl m Z m Z ddl m Z ddlmZddlmZdd lmZdd lmZmZmZd d gZGd ddeeZGdd d eZGdd d eZdS)N)sparse) BaseEstimatorTransformerMixin) check_array is_scalar_nan) deprecated)check_is_fitted)_check_feature_names_in) _get_mask)_encode_check_unknown_unique OneHotEncoderOrdinalEncoderc@s>eZdZdZdddZddZddd Zdd d Zd dZdS) _BaseEncoderzm Base class for encoders that includes the code to categorize and transform the input features. Tc Cst|drt|dddks\t|d|d}t|dsRt|jtjrRt|t|d}n|}d}n|}|j\}}g}t |D],}|j ||d } t| dd|d } | | qv|||fS) a Perform custom check_array: - convert list of strings to object dtype - check for missing values for object dtype data (check_array does not do that) - return list of features (arrays): this list of features is constructed feature by feature to preserve the data types of pandas DataFrame columns, as otherwise information is lost and cannot be used, e.g. for the `categories_` attribute. ilocndimrrN)dtypeforce_all_finiterF) feature_idx)Z ensure_2drr) hasattrgetattrrnpZ issubdtyperZstr_objectshaperange _get_featureappend) selfXrZX_tempZneeds_validation n_samples n_featuresZ X_columnsiXir%>lib/python3.9/site-packages/sklearn/preprocessing/_encoders.py_check_Xs    z_BaseEncoder._check_XcCs,t|dr|jdd|fS|dd|fS)Nr)rr)rr rr%r%r&rCs z_BaseEncoder._get_featureerrorcCsT|j|dd|j|dd|j||d\}}}||_|jdkrVt|j|krVtdg|_t|D]}||}|jdkrt |} nt j |j||j d} |j j dvrt | } d} t | d rd nd} t | d| | d| ks t | d rt | d st| |d krBt|| } | rBd | |}t||j| qddS) NTresetrautozOShape mismatch: if categories is an array, it has to be of shape (n_features,).rZOUSz>Unsorted categories are not supported for numerical categoriesr(z5Found unknown categories {0} in column {1} during fit)_check_n_features_check_feature_namesr'Zn_features_in_ categorieslen ValueError categories_rrrarrayrkindsortZisnananyr formatr)rr handle_unknownrX_listr!r"r#r$catsZ sorted_catsZ error_msgZstop_idxdiffmsgr%r%r&_fitJsJ           z_BaseEncoder._fitFcCs|j|dd|j|dd|j||d\}}}tj||ftd}tj||ftd} g} t|D]} || } t | |j | dd\} }t |sT|dkrd | | }t |n|r| | || dd| f<|j | jjd vr |j | j| jkr | |j | j} n6|j | jjd kr8| jjd kr8| d } n| } |j | d | |<t| |j | dd |dd| f<q`| rtd| dt|| fS)NFr)r+r-T)Z return_maskr(z;Found unknown categories {0} in column {1} during transform)USOr@r)ZuniquesZ check_unknownz$Found unknown categories in columns zH during transform. These unknown categories will be encoded as all zeros)r0r/r'rzerosintonesboolrr r4allr9r3rrr6itemsizeastypecopyr warningswarn UserWarning)rr r:rwarn_on_unknownr;r!r"X_intX_maskZcolumns_with_unknownr#r$r=Z valid_maskr>r%r%r& _transformwsN    " "z_BaseEncoder._transformcCs ddgiS)NZX_typesZ categoricalr%)rr%r%r& _more_tagssz_BaseEncoder._more_tagsN)T)r(T)r(TF) __name__ __module__ __qualname____doc__r'rr?rQrRr%r%r%r&rs % . `. Parameters ---------- categories : 'auto' or a list of array-like, default='auto' Categories (unique values) per feature: - 'auto' : Determine categories automatically from the training data. - list : ``categories[i]`` holds the categories expected in the ith column. The passed categories should not mix strings and numeric values within a single feature, and should be sorted in case of numeric values. The used categories can be found in the ``categories_`` attribute. .. versionadded:: 0.20 drop : {'first', 'if_binary'} or an array-like of shape (n_features,), default=None Specifies a methodology to use to drop one of the categories per feature. This is useful in situations where perfectly collinear features cause problems, such as when feeding the resulting data into a neural network or an unregularized regression. However, dropping one category breaks the symmetry of the original representation and can therefore induce a bias in downstream models, for instance for penalized linear classification or regression models. - None : retain all features (the default). - 'first' : drop the first category in each feature. If only one category is present, the feature will be dropped entirely. - 'if_binary' : drop the first category in each feature with two categories. Features with 1 or more than 2 categories are left intact. - array : ``drop[i]`` is the category in feature ``X[:, i]`` that should be dropped. .. versionadded:: 0.21 The parameter `drop` was added in 0.21. .. versionchanged:: 0.23 The option `drop='if_binary'` was added in 0.23. sparse : bool, default=True Will return sparse matrix if set True else will return an array. dtype : number type, default=float Desired dtype of output. handle_unknown : {'error', 'ignore'}, default='error' Whether to raise an error or ignore if an unknown categorical feature is present during transform (default is to raise). When this parameter is set to 'ignore' and an unknown category is encountered during transform, the resulting one-hot encoded columns for this feature will be all zeros. In the inverse transform, an unknown category will be denoted as None. Attributes ---------- categories_ : list of arrays The categories of each feature determined during fitting (in order of the features in X and corresponding with the output of ``transform``). This includes the category specified in ``drop`` (if any). drop_idx_ : array of shape (n_features,) - ``drop_idx_[i]`` is the index in ``categories_[i]`` of the category to be dropped for each feature. - ``drop_idx_[i] = None`` if no category is to be dropped from the feature with index ``i``, e.g. when `drop='if_binary'` and the feature isn't binary. - ``drop_idx_ = None`` if all the transformed features will be retained. .. versionchanged:: 0.23 Added the possibility to contain `None` values. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 1.0 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- OrdinalEncoder : Performs an ordinal (integer) encoding of the categorical features. sklearn.feature_extraction.DictVectorizer : Performs a one-hot encoding of dictionary items (also handles string-valued features). sklearn.feature_extraction.FeatureHasher : Performs an approximate one-hot encoding of dictionary items or strings. LabelBinarizer : Binarizes labels in a one-vs-all fashion. MultiLabelBinarizer : Transforms between iterable of iterables and a multilabel format, e.g. a (samples x classes) binary matrix indicating the presence of a class label. Examples -------- Given a dataset with two features, we let the encoder find the unique values per feature and transform the data to a binary one-hot encoding. >>> from sklearn.preprocessing import OneHotEncoder One can discard categories not seen during `fit`: >>> enc = OneHotEncoder(handle_unknown='ignore') >>> X = [['Male', 1], ['Female', 3], ['Female', 2]] >>> enc.fit(X) OneHotEncoder(handle_unknown='ignore') >>> enc.categories_ [array(['Female', 'Male'], dtype=object), array([1, 2, 3], dtype=object)] >>> enc.transform([['Female', 1], ['Male', 4]]).toarray() array([[1., 0., 1., 0., 0.], [0., 1., 0., 0., 0.]]) >>> enc.inverse_transform([[0, 1, 1, 0, 0], [0, 0, 0, 1, 0]]) array([['Male', 1], [None, 2]], dtype=object) >>> enc.get_feature_names_out(['gender', 'group']) array(['gender_Female', 'gender_Male', 'group_1', 'group_2', 'group_3'], ...) One can always drop the first column for each feature: >>> drop_enc = OneHotEncoder(drop='first').fit(X) >>> drop_enc.categories_ [array(['Female', 'Male'], dtype=object), array([1, 2, 3], dtype=object)] >>> drop_enc.transform([['Female', 1], ['Male', 2]]).toarray() array([[0., 0., 0.], [1., 1., 0.]]) Or drop a column for feature only having 2 categories: >>> drop_binary_enc = OneHotEncoder(drop='if_binary').fit(X) >>> drop_binary_enc.transform([['Female', 1], ['Male', 2]]).toarray() array([[0., 1., 0., 0.], [1., 0., 1., 0.]]) r,NTr()r1droprrr:cCs"||_||_||_||_||_dSN)r1rrr:rW)rr1rWrrr:r%r%r&__init__[s zOneHotEncoder.__init__cCs"|jdvrd|j}t|dS)N)r(ignorez=handle_unknown should be either 'error' or 'ignore', got {0}.)r:r9r3)rr>r%r%r&_validate_keywordsjs  z OneHotEncoder._validate_keywordsc Cs|jdurdSt|jtrx|jdkr8tjt|jtdS|jdkr\tjdd|jDtdSd}t | t |jnVztj |jtd}t|}Wn,t t fyd}t | t |Yn0|t|jkrd}t | t|j|g}g}tt||jD]\}\}}t|sTt||kd} | jrB|| dn|||fqt|D]$\} } t| r\|| qq\|||fqt|rd d d d|D}t |tj|tdSdS) Nfirstr-Z if_binarycSs g|]}t|dkrdndqS)rrNr2.0r<r%r%r& {z3OneHotEncoder._compute_drop_idx..zaWrong input for parameter `drop`. Expected 'first', 'if_binary', None or array of objects, got {}zF`drop` should have length equal to the number of features ({}), got {}rzaThe following categories were supposed to be dropped, but were not found in the training data. {} cSsg|]\}}d||qS)zCategory: {}, Feature: {})r9)r_cvr%r%r&r`s)rW isinstancestrrrCr2r4rr5r3r9typeasarray TypeError enumerateziprwheresizerr8join) rr>Z drop_arrayZdroplenZ missing_dropsZ drop_indicesZcol_idxvalZcat_listZdrop_idxcat_idxcatr%r%r&_compute_drop_idxssf            zOneHotEncoder._compute_drop_idxcCs(||j||jdd||_|S)a Fit OneHotEncoder to X. Parameters ---------- X : array-like of shape (n_samples, n_features) The data to determine the categories of each feature. y : None Ignored. This parameter exists only for compatibility with :class:`~sklearn.pipeline.Pipeline`. Returns ------- self Fitted encoder. allow-nanr:r)r[r?r:rr drop_idx_rr yr%r%r&fits zOneHotEncoder.fitcs|t||S)af Fit OneHotEncoder to X, then transform X. Equivalent to fit(X).transform(X) but more convenient. Parameters ---------- X : array-like of shape (n_samples, n_features) The data to encode. y : None Ignored. This parameter exists only for compatibility with :class:`~sklearn.pipeline.Pipeline`. Returns ------- X_out : {ndarray, sparse matrix} of shape (n_samples, n_encoded_features) Transformed input. If `sparse=True`, a sparse matrix will be returned. )r[super fit_transformrv __class__r%r&rzszOneHotEncoder.fit_transformcCst||jdko|jdu}|j||jd|d\}}|j\}}|jdur|j}||k}g} t|jD]>\} } t | } || dur| || <| | qh| | dqh| dd}|||kd8<||M}ndd|jD} | } t d g| }||dd | }t j|dtd }d |d <t j|d|dd|jd t j|dd|ddd t |d}tj|||f||df|jd }|js|S|SdS)a Transform X using one-hot encoding. Parameters ---------- X : array-like of shape (n_samples, n_features) The data to encode. Returns ------- X_out : {ndarray, sparse matrix} of shape (n_samples, n_encoded_features) Transformed input. If `sparse=True`, a sparse matrix will be returned. rZNrs)r:rrNr.cSsg|] }t|qSr%r]r^r%r%r&r`raz+OneHotEncoder.transform..rr-)axisoutr)r)rr)r r:rWrQrrurJrjr4r2rZreshapeZravelrZcumsumemptyrDsumrrErZ csr_matrixZtoarray)rr rNrOrPr!r"to_dropZ keep_cellsZn_valuesr#r<Zn_catsmaskZfeature_indicesindicesZindptrdatarr%r%r& transformsN         zOneHotEncoder.transformcCst|t|dd}|j\}}t|j}|jdurHtdd|jD}ntddt|j|jD}d}|jd|krt| ||jdt d d |jDg}t j ||f|d }d } i} t |D]} |jdus|j| dur|j| } nt |j| |j| } t| } | d krD|j| |j| |dd| f<| | 7} q|dd| | | f}t |jdd }| ||dd| f<|jdkrt |jdd d k}|r`|jdus|j| dur|| | <n|j| |j| ||| f<nft |jdd d k}|r`|jdurDt |}td|d|j| |j| ||| f<| | 7} q| r|jtkr|t}| D]\}}d|||f<q|S)aw Convert the data back to the original representation. When unknown categories are encountered (all zeros in the one-hot encoding), ``None`` is used to represent this category. If the feature with the unknown category has a dropped category, the dropped category will be its inverse. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_encoded_features) The transformed data. Returns ------- X_tr : ndarray of shape (n_samples, n_features) Inverse transformed array. Zcsr)Z accept_sparseNcss|]}t|VqdSrXr]r^r%r%r& Kraz2OneHotEncoder.inverse_transform..css.|]&\}}|durt|dnt|VqdS)Nr}r])r_r<rr%r%r&rMsIShape of the passed X data is not correct. Expected {0} columns, got {1}.r}cSsg|] }|jqSr%r-r_rqr%r%r&r`Zraz3OneHotEncoder.inverse_transform..r-r)r~rZzSamples z] can not be inverted when drop=None and handle_unknown='error' because they contain all zeros)r rrr2r4rurrkr3r9rfind_common_typerrdeleterhZargmaxZflattenr:r8Z flatnonzerorrrIitems)rr r!_r"Zn_transformed_featuresr>dtX_trj found_unknownr#r<Z n_categoriessublabelsunknownZdroppedZall_zero_samplesidxrr%r%r&inverse_transform1sd                  zOneHotEncoder.inverse_transformzlget_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead.cst||j}dur.ddtt|Dn,tt|jkrZtdt|jtg}tt|D]Nfdd|D}|jdur|jdur||j||qjt j |t dS)aReturn feature names for output features. Parameters ---------- input_features : list of str of shape (n_features,) String names for input features if available. By default, "x0", "x1", ... "xn_features" is used. Returns ------- output_feature_names : ndarray of shape (n_output_features,) Array of feature names. NcSsg|] }d|qS)zx%dr%)r_r#r%r%r&r`raz3OneHotEncoder.get_feature_names..zJinput_features should have length equal to number of features ({}), got {}cs g|]}dt|qSrrfr_tr#input_featuresr%r&r`rar-) r r4rr2r3r9rupopextendrr5rrrr<Z feature_namesnamesr%rr&get_feature_namess" zOneHotEncoder.get_feature_namescst||j}t|g}tt|D]Nfdd|D}|jdurl|jdurl||j||q(tj |t dS)aGet output feature names for transformation. Parameters ---------- input_features : array-like of str or None, default=None Input features. - If `input_features` is `None`, then `feature_names_in_` is used as feature names in. If `feature_names_in_` is not defined, then names are generated: `[x0, x1, ..., x(n_features_in_)]`. - If `input_features` is an array-like, then `input_features` must match `feature_names_in_` if `feature_names_in_` is defined. Returns ------- feature_names_out : ndarray of str objects Transformed feature names. cs g|]}dt|qSrrrrr%r&r`raz7OneHotEncoder.get_feature_names_out..Nr-) r r4r rr2rurrrrhrrr%rr&get_feature_names_outs  z#OneHotEncoder.get_feature_names_out)N)N)N)N)rSrTrUrVrfloat64rYr[rrrxrzrrrrr __classcell__r%r%r{r&rs&'  G Gf #c@s@eZdZdZdejdddddZddd Zd d Zd d Z dS)raO Encode categorical features as an integer array. The input to this transformer should be an array-like of integers or strings, denoting the values taken on by categorical (discrete) features. The features are converted to ordinal integers. This results in a single column of integers (0 to n_categories - 1) per feature. Read more in the :ref:`User Guide `. .. versionadded:: 0.20 Parameters ---------- categories : 'auto' or a list of array-like, default='auto' Categories (unique values) per feature: - 'auto' : Determine categories automatically from the training data. - list : ``categories[i]`` holds the categories expected in the ith column. The passed categories should not mix strings and numeric values, and should be sorted in case of numeric values. The used categories can be found in the ``categories_`` attribute. dtype : number type, default np.float64 Desired dtype of output. handle_unknown : {'error', 'use_encoded_value'}, default='error' When set to 'error' an error will be raised in case an unknown categorical feature is present during transform. When set to 'use_encoded_value', the encoded value of unknown categories will be set to the value given for the parameter `unknown_value`. In :meth:`inverse_transform`, an unknown category will be denoted as None. .. versionadded:: 0.24 unknown_value : int or np.nan, default=None When the parameter handle_unknown is set to 'use_encoded_value', this parameter is required and will set the encoded value of unknown categories. It has to be distinct from the values used to encode any of the categories in `fit`. If set to np.nan, the `dtype` parameter must be a float dtype. .. versionadded:: 0.24 Attributes ---------- categories_ : list of arrays The categories of each feature determined during ``fit`` (in order of the features in X and corresponding with the output of ``transform``). This does not include categories that weren't seen during ``fit``. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 1.0 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- OneHotEncoder : Performs a one-hot encoding of categorical features. LabelEncoder : Encodes target labels with values between 0 and ``n_classes-1``. Examples -------- Given a dataset with two features, we let the encoder find the unique values per feature and transform the data to an ordinal encoding. >>> from sklearn.preprocessing import OrdinalEncoder >>> enc = OrdinalEncoder() >>> X = [['Male', 1], ['Female', 3], ['Female', 2]] >>> enc.fit(X) OrdinalEncoder() >>> enc.categories_ [array(['Female', 'Male'], dtype=object), array([1, 2, 3], dtype=object)] >>> enc.transform([['Female', 3], ['Male', 1]]) array([[0., 2.], [1., 0.]]) >>> enc.inverse_transform([[1, 0], [0, 1]]) array([['Male', 1], ['Female', 2]], dtype=object) r,r(Nr1rr: unknown_valuecCs||_||_||_||_dSrXr)rr1rr:rr%r%r&rY9szOrdinalEncoder.__init__c Csnd}|j|vr td|jd|jdkr|t|jrZt|jjdkrztd|jdqt|jtj st d|jdn|jdurt d |jd|j ||jd d |jdkr|j D]4}d |jkrt |krnqtd |jdqi|_t|j D]6\}}t|D]"\}}t|r||j|<qqqt|jjdkrj|jrjtdt|jd|S)a Fit the OrdinalEncoder to X. Parameters ---------- X : array-like of shape (n_samples, n_features) The data to determine the categories of each feature. y : None Ignored. This parameter exists only for compatibility with :class:`~sklearn.pipeline.Pipeline`. Returns ------- self : object Fitted encoder. )r(use_encoded_valuezDhandle_unknown should be either 'error' or 'use_encoded_value', got .rfzOWhen unknown_value is np.nan, the dtype parameter should be a float dtype. Got z]unknown_value should be an integer or np.nan when handle_unknown is 'use_encoded_value', got NzQunknown_value should only be set when handle_unknown is 'use_encoded_value', got rsrtrz!The used value for unknown_value zD is one of the values already used for encoding the seen categories.z%There are missing values in features zW. For OrdinalEncoder to passthrough missing values, the dtype parameter must be a float)r:r3rrrrr6renumbersZIntegralrir?r4r2_missing_indicesrjlist) rr rwZhandle_unknown_strategiesZ feature_catsrpZcategories_for_idxr#rqr%r%r&rxFsb         zOrdinalEncoder.fitcCsv|j||jdd\}}|j|jdd}|jD]*\}}|dd|f|k}tj|||f<q0|jdkrr|j||<|S)a' Transform X to ordinal codes. Parameters ---------- X : array-like of shape (n_samples, n_features) The data to encode. Returns ------- X_out : ndarray of shape (n_samples, n_features) Transformed input. rsrtFrJNr) rQr:rIrrrrnanr)rr rOrPZX_transrpZ missing_idxZX_missing_maskr%r%r&rs   zOrdinalEncoder.transformcCslt|t|dd}|j\}}t|j}d}|jd|krPt|||jdtdd|jDg}tj ||f|d}i}t |D]} |dd| fj d d d } | |j vrt |dd| ftj} |j | | | <|jd kr| |jk} |j| t| d | |dd| f<| || <q|j| | |dd| f<q|rh|j td d }|D]\} }d||| f<qP|S)aP Convert the data back to the original representation. Parameters ---------- X : array-like of shape (n_samples, n_encoded_features) The transformed data. Returns ------- X_tr : ndarray of shape (n_samples, n_features) Inverse transformed array. rsr+rr}cSsg|] }|jqSr%r-rr%r%r&r`raz4OrdinalEncoder.inverse_transform..r-NZint64Frrr)r rrr2r4r3r9rrrrrIrr rr:rrlrr)rr r!rr"r>rrrr#rZX_i_maskZunknown_labelsrrr%r%r&rs4       $ z OrdinalEncoder.inverse_transform)N) rSrTrUrVrrrYrxrrr%r%r%r&rs] N)rKZnumpyrZscipyrrbaserrZutilsrrZutils.deprecationrZutils.validationr r Z utils._maskr Z utils._encoder r r__all__rrrr%r%r%r&s&     !+