Ae/ddlmZddlZejeeeeddlmZddlmZm Z m Z eZ GddeZ y) )absolute_importN)BOLEOLEOF NOT_FOUND)Errors)rrrcPeZdZdZd dZdZdZdZdZdZ d Z d Z dd Z d Z y)Scannera A Scanner is used to read tokens from a stream of characters using the token set specified by a Plex.Lexicon. Constructor: Scanner(lexicon, stream, name = '') See the docstring of the __init__ method for details. Methods: See the docstrings of the individual methods for more information. read() --> (value, text) Reads the next lexical token from the stream. position() --> (name, line, col) Returns the position of the last token read using the read() method. begin(state_name) Causes scanner to change state. produce(value [, text]) Causes return of a token value to the caller of the Scanner. Ncd|_d|_d|_d|_d|_d|_d|_d|_d|_d|_ d|_ ||_ ||_ ||_ g|_d|_|j!dd|_d|_d|_t$|_d|_||d|d c|_|_yy)a Scanner(lexicon, stream, name = '') |lexicon| is a Plex.Lexicon instance specifying the lexical tokens to be recognised. |stream| can be a file object or anything which implements a compatible read() method. |name| is optional, and may be the name of the file being scanned or any other identifying string. rrN)tracebuffer buf_start_posnext_poscur_poscur_line start_pos start_line start_coltext state_namelexiconstreamnamequeue initial_statebegincur_line_startrcur_char input_state)selfrrr initial_poss 4lib/python3.12/site-packages/Cython/Plex/Scanners.py__init__zScanner.__init__Is          ! 2     "1::;;;!3!33 t1113D&> !||t~~-==C'NN$==(DMMS,@&**4A Ar'c |j}|j}|j}|j}|j}|j }|j }|j}|j} t|} d\} } } }}}}|j} |rtd|d||t|fz|d}||||||||f\} } } }}}}|}|j|t}|tur|xr|jd}|r|rtd|dz|}|dk(r|}|| z }|| kr ||}|dz }nn|j| z }|j j#d }|j|d|z}||_| |z } | |_t|} ||z}|r ||}|dz }nd }|d k(r t$}d }nd|s t$}d }nY|}nV|d k(rd }d}nL|dk(r|dz }|x}}t&}d}n5|d k(r t(}d}n'd }n$|r td| | | | ||||f\}}}}}}}nd}n||_||_||_||_||_||_|r|td|z|S)z; Inlined version of run_machine for speed. )Nrrrr rrrzState %d, %d/%d:%s -->numberr-NelsezState %dir  rblockedzDoing %s)rrrr r!r"rrrlenrr4reprgetrrrr0rrr)r#staterrr r!r"rrrbuf_lenb_action b_cur_pos b_cur_lineb_cur_line_start b_cur_char b_input_state b_next_posrr-c new_state buf_indexdiscarddatas r%r3zScanner.run_machine_inlineds"",,==,,==&& ==** f+ $ a)Z)9:}V` .(O['4>2KKL8_F!GX~xV^^i)Z1A:}^hA !Y/II%3%))F"3 *y'::;!!#&G (= 8I 7*"9- A "&..="@#{{//7!%WX!6!=&, %0 -:*"%f+!W,  &y 1A$MH #AEz#&&' #&&' #$ A%$H"#K A%MH/77NW"H"#K A%"H"#K"H)$'"9j:J#]J@6VWh{H"FY\   ,  &  !j6)* r'ct|j}|jrtdd||jfz|dk(rV|j|_|j }|dk(rt |_d|_n|st |_d|_n||_nx|dk(rd|_d|_nd|dk(r@|xjdz c_|jx|_ |_t|_d|_n|dk(rt|_d|_nd |_|jr&td ||j|jfzyy) NzScanner: next: %s [%d] %dz rr:rr;r<r=r z--> [%d] %d %r) r"rr4rr read_charrr!rr rr)r#r"rKs r%r5zScanner.next_char s&& :: -; 0UU V ! ==DL AEz # #$  # #$ ! A !DM D  A  MMQ M15 >D $,DM D  A DM D DM :: "k4<<%OO P r'cH|j|j|jfS)am Return a tuple (name, line, col) representing the location of the last token read using the read() method. |name| is the name that was provided to the Scanner constructor; |line| is the line number in the stream (1-based); |col| is the position within the line of the first character of the token (0-based). )rrrr#s r%positionzScanner.position)s 4??DNN;;r'c"|jS)zOPython accessible wrapper around position(), only for error reporting. )rTrSs r% get_positionzScanner.get_position4s}}r'cR|jj||_||_y)z8Set the current state of the scanner to the named state.N)rget_initial_staterr)r#rs r%rz Scanner.begin9s% LL * *: 6 $r'cZ| |j}|jj||fy)a Called from an action procedure, causes |value| to be returned as the token value from read(). If |text| is supplied, it is returned in place of the scanned text. produce() can be called more than once during a single call to an action procedure, in which case the tokens are queued up and returned one at a time by subsequent calls to read(), until the queue is empty, whereupon scanning resumes. N)rrappend)r#r.rs r%r*zScanner.produce?s( <99D 5$-(r'cy)z_ Override this method if you want something to be done at end of file. NrSs r%r+z Scanner.eofNsr'r2)N)__name__ __module__ __qualname____doc__r&r0r)r3r5rTrVrr*r+r\r'r%r r s@b&QP*B4hTQ> < % ) r'r ) __future__rcythondeclareobjectr r Regexpsrrrrr r\r'r%rfs?' 6v6VD"" H { f{ r'