a GGb@s\ddlmZddlmZddlmZddlmZddlm Z ddl m Z Gddde Z d S) ) annotations) ReadBuffer)import_optional_dependency) is_integer) DataFrame) ParserBasecsTeZdZdZddfdd ZddZdd Zd d d d d Zd dddZZ S)ArrowParserWrapperz7 Wrapper for the pyarrow engine for read_csv() zReadBuffer[bytes])srcc s$t|||_||_|dS)N)super__init__kwdsr _parse_kwds)selfr r  __class__Elib/python3.9/site-packages/pandas/io/parsers/arrow_parser_wrapper.pyr s zArrowParserWrapper.__init__cCsf|jd}|durdn||_||jd\|_|_|jd}t|trRtdt |jd|_ dS)z? Validates keywords before passing to pyarrow. encodingNzutf-8usecols na_valuesz?The pyarrow engine doesn't support passing a dict for na_values) r getrZ_validate_usecols_argrZ usecols_dtype isinstancedict ValueErrorlistr)rrrrrrr s    zArrowParserWrapper._parse_kwdscCsddddd}|D]4\}}||jvr|j|dur|j||j|<qdd|jD|_d d|jD|_|jdu|jdur|jn|jd d |_dS) z: Rename some arguments to pass to pyarrow include_columns null_values escape_charignore_empty_lines)rrZ escapecharZskip_blank_linesNcSs&i|]\}}|dur|dvr||qS)N)Z delimiterZ quote_charrrr.0Z option_nameZ option_valuerrr 8s z;ArrowParserWrapper._get_pyarrow_options..cSs&i|]\}}|dur|dvr||qS)N)rrZ true_valuesZ false_valuesrrrrrr!?s Zskiprows)Zautogenerate_column_namesZ skip_rows)itemsr rpop parse_optionsconvert_optionsheader read_options)rmappingZ pandas_nameZ pyarrow_namerrr_get_pyarrow_options*s&z'ArrowParserWrapper._get_pyarrow_optionsr)framereturnc st|j}d}jdurjdurZjdurFfddt|D_njdurZt|_tj|krtt|tjj_d}j|_|j|\|_}jdur4t jD]<\}}t |r|j|j|<q||jvrt d|dq|j jdddjdur4|s4dgt|j j|j _jd durz|jd }Wn.ty}zt |WYd}~n d}~00|S) z Processes data read in based on kwargs. Parameters ---------- frame: DataFrame The DataFrame to process. Returns ------- DataFrame The processed DataFrame. TNcsg|]}j|qSr)prefix)r irrr `z7ArrowParserWrapper._finalize_output..FzIndex z invalid)ZdropZinplaceZdtype)lencolumnsr&namesr,rangerZ_do_date_conversionsZ index_col enumeraterrZ set_indexindexr rZastype TypeError)rr*Znum_colsZmulti_index_namedr-itemerr.r_finalize_outputMs<         z#ArrowParserWrapper._finalize_output)r+cCs`td}||j|j|jfi|j|jfi|j|jfi|j d}| }| |S)z Reads the contents of a CSV file into a DataFrame and processes it according to the kwargs passed in the constructor. Returns ------- DataFrame The DataFrame created from the CSV file. z pyarrow.csv)r'r$r%) rr)Zread_csvr Z ReadOptionsr'Z ParseOptionsr$ZConvertOptionsr%Z to_pandasr:)rZ pyarrow_csvtabler*rrrreads zArrowParserWrapper.read) __name__ __module__ __qualname____doc__r r r)r:r< __classcell__rrrrr s #:rN) Z __future__rZpandas._typingrZpandas.compat._optionalrZpandas.core.dtypes.inferencerZpandas.core.framerZpandas.io.parsers.base_parserrrrrrrs