a GGb'@sNdZddlmZddlZddlmZmZmZmZm Z ddl Z ddl m Z ddlmZddlmZddlmZmZdd lmZmZmZmZerdd lmZmZmZmZmZd Z d d ddddZ!dde dfdddddddddZ"de fdddddddZ#d dddd!d"d#Z$de dfd$ddddd%d&d'Z%de dfdddddd%d(d)Z&dS)*z" data hash pandas / numpy objects ) annotationsN) TYPE_CHECKINGHashableIterableIteratorcast)lib)hash_object_array) ArrayLike)is_categorical_dtype is_list_like) ABCDataFrameABCIndex ABCMultiIndex ABCSeries) Categorical DataFrameIndex MultiIndexSeriesZ0123456789123456zIterator[np.ndarray]intz np.ndarray)arrays num_itemsreturncCsz t|}Wn"ty.tjgtjdYS0t|g|}td}t|td}t|D]6\}}||}||N}||9}|td||7}qd|d|ksJd|td7}|S)z Parameters ---------- arrays : Iterator[np.ndarray] num_items : int Returns ------- np.ndarray[uint64] Should be the same as CPython's tupleobject.c dtypeiCBixV4iXBzFed in wrong num_itemsi|) next StopIterationnpZarrayuint64 itertoolschainZ zeros_like enumerate)rrfirstZmultoutiaZ inverse_ir(7lib/python3.9/site-packages/pandas/core/util/hashing.pycombine_hash_arrays.s    r*Tutf8zIndex | DataFrame | Seriesboolstrz str | Noner)objindexencodinghash_key categorizercsddlm}durtttr8|tdddSttrptjj ddd}||ddd}n tt rtjj ddd}|rȇfd d d D}t |g|} t | d }||jddd}nttrjfd d D} tj} |rLfdd d D} | d7} t | | } dd | D} t | | }||jddd}ntdt|S)a~ Return a data hash of the Index/Series/DataFrame. Parameters ---------- obj : Index, Series, or DataFrame index : bool, default True Include the index in the hash (if Series/DataFrame). encoding : str, default 'utf8' Encoding for data & key when strings. hash_key : str, default _default_hash_key Hash_key for string key to encode. categorize : bool, default True Whether to first categorize object arrays before hashing. This is more efficient when the array contains duplicate values. Returns ------- Series of uint64, same length as the object r)rNr F)rcopyr3)r/rr3c3s$|]}tjddjVqdSF)r/r0r1r2Nhash_pandas_objectr/_values.0_r2r0r1r.r(r) |sz%hash_pandas_object..Nc3s"|]\}}t|jVqdSr>) hash_arrayr8)r:r;Zseries)r2r0r1r(r)r=sc3s$|]}tjddjVqdSr5r6r9r<r(r)r=srcss|] }|VqdSr>r()r:xr(r(r)r=zUnexpected type for hashing )pandasr_default_hash_key isinstancer hash_tuplesrr@r8astyperr!r"r*r/r itemslencolumns TypeErrortype)r.r/r0r1r2rhZserZ index_iterrhashesrZindex_hash_generatorZ_hashesr(r<r)r7NsJ           r7z+MultiIndex | Iterable[tuple[Hashable, ...]])valsr0r1rcszt|stdddlmm}t|ts6||n|fddtj D}fdd|D}t |t |}|S)a Hash an MultiIndex / listlike-of-tuples efficiently. Parameters ---------- vals : MultiIndex or listlike-of-tuples encoding : str, default 'utf8' hash_key : str, default _default_hash_key Returns ------- ndarray[np.uint64] of hashed values z'must be convertible to a list-of-tuplesr)rrcs(g|] }j|j|dddqS)FTZorderedZfastpath)codesZlevels)r:level)rmir(r) szhash_tuples..c3s|]}t|dVqdS)r0r1N)_hash_categorical)r:catrUr(r)r=szhash_tuples..) r rKrCrrrErZ from_tuplesrangeZnlevelsr*rI)rOr0r1rZcat_valsrNrMr()rr0r1rSr)rFs    rFr)rWr0r1rcCsdt|jj}t|||dd}|}t|r<||j}ntj t|dd}| r`t j ||<|S)a Hash a Categorical by hashing its categories, and then mapping the codes to the hashes Parameters ---------- cat : Categorical encoding : str hash_key : str Returns ------- ndarray[np.uint64] of hashed values, same size as len(c) F)r2r r) rZasarray categoriesr8r@ZisnarIZtakerQZzerosanyrZu8max)rWr0r1valuesZhashedmaskresultr(r(r)rVs  rVr )rOr0r1r2rcCs\t|dstd|j}t|r6td|}t|||St|tjsN| \}}t ||||S)aK Given a 1d array, return an array of deterministic integers. Parameters ---------- vals : ndarray or ExtensionArray encoding : str, default 'utf8' Encoding for data & key when strings. hash_key : str, default _default_hash_key Hash_key for string key to encode. categorize : bool, default True Whether to first categorize object arrays before hashing. This is more efficient when the array contains duplicate values. Returns ------- ndarray[np.uint64, ndim=1] Hashed values, same length as the vals. rzmust pass a ndarray-liker) hasattrrKrr rrVrErZndarrayZ_values_for_factorize _hash_ndarray)rOr0r1r2rr;r(r(r)r@s     r@c Cst|j}t|tjr4tt|dtt|St|trJ| d}nt |j tj tj frt|dj ddd}nt |j tjr|jdkr|d|jj d}n|rdd lm}m}m}||dd \}} |||| dd d } t| ||Szt|||}Wn,ty.t| t t||}Yn0||d ?N}|td9}||d?N}|td9}||d?N}|S)z! See hash_array.__doc__. u8Zi8Fr4ur)rr factorize)sortTrPle9z lb&&& )rrZ issubdtypeZ complex128r@realimagrEr,rG issubclassrLZ datetime64Z timedelta64ZviewZnumberitemsizerCrrrdZ _with_inferrVr rKr-objectr ) rOr0r1r2rrrrdrQrYrWr(r(r)r_(s8         r_)'__doc__Z __future__rr!typingrrrrrZnumpyrZ pandas._libsrZpandas._libs.hashingr Zpandas._typingr Zpandas.core.dtypes.commonr r Zpandas.core.dtypes.genericr rrrrCrrrrrrDr*r7rFrVr@r_r(r(r(r)s<     "^.(,