a &b 9@sdZddlZddlZddlmZmZddlmZmZm Z m Z ddl m Z m Z GdddeZGd d d eZGd d d ejZeje d dee dddZddZdddZGdddeZGdddeZdS)z XPath selectors based on lxml N)etreehtml)flatteniflatten extract_regexshorten)HTMLTranslatorGenericTranslatorc@s eZdZdS)CannotRemoveElementWithoutRootN__name__ __module__ __qualname__rr.lib/python3.9/site-packages/parsel/selector.pyr sr c@s eZdZdS) CannotRemoveElementWithoutParentNr rrrrrsrcseZdZfddZZS) SafeXMLParsercs&|ddtt|j|i|dS)NZresolve_entitiesF) setdefaultsuperr__init__)selfargskwargs __class__rrrs zSafeXMLParser.__init__)r rrr __classcell__rrrrrsrr)_parser_csstranslator_tostring_methodxml)rr cCs(|dur dS|tvr|Std|dS)NrzInvalid type: %s)_ctgroup ValueError)strrr_st&s r$cCsR|dddpd}|ddd}tj|||d}|durNtjd||d}|S) z8Create root node for text using given parser class. utf8sT)Zrecoverencoding)parserbase_urlN)stripreplaceencoderZ fromstring)textZ parser_clsr*bodyr)rootrrrcreate_root_node/s  r1cseZdZdZfddZfddZddZdd d Zd d ZdddZ dddZ ddZ e Z dddZ e ZeddZddZZS) SelectorListz The :class:`SelectorList` class is a subclass of the builtin ``list`` class, which provides a few additional methods. cstt|||}||SN)rr2 __getslice__r)rijorrrr4AszSelectorList.__getslice__cs(tt||}t|tr$||S|Sr3)rr2 __getitem__ isinstanceslicer)rposr7rrrr8EszSelectorList.__getitem__cCs tddS)Nz!can't pickle SelectorList objects TypeErrorrrrr __getstate__IszSelectorList.__getstate__Nc s |tfdd|DS)a Call the ``.xpath()`` method for each element in this list and return their results flattened as another :class:`SelectorList`. ``query`` is the same argument as the one in :meth:`Selector.xpath` ``namespaces`` is an optional ``prefix: namespace-uri`` mapping (dict) for additional prefixes to those registered with ``register_namespace(prefix, uri)``. Contrary to ``register_namespace()``, these prefixes are not saved for future calls. Any additional named arguments can be used to pass values for XPath variables in the XPath expression, e.g.:: selector.xpath('//a[href=$url]', url="http://www.example.com") cs"g|]}|jfdiqS namespaces)xpath.0xrrArBrr ]z&SelectorList.xpath..rr)rrBrArrrFrrBLszSelectorList.xpathcs|tfdd|DS)z Call the ``.css()`` method for each element in this list and return their results flattened as another :class:`SelectorList`. ``query`` is the same argument as the one in :meth:`Selector.css` csg|]}|qSr)cssrCqueryrrrGfrHz$SelectorList.css..rIrrLrrKrrJ_szSelectorList.cssTcstfdd|DS)ay Call the ``.re()`` method for each element in this list and return their results flattened, as a list of unicode strings. By default, character entity references are replaced by their corresponding character (except for ``&`` and ``<``. Passing ``replace_entities`` as ``False`` switches off these replacements. csg|]}|jdqS)replace_entitiesrerCregexrOrrrGrrHz#SelectorList.re..)rrrSrOrrRrrQhs zSelectorList.recs(tfdd|DD] }|S|S)a Call the ``.re()`` method for the first element in this list and return the result in an unicode string. If the list is empty or the regex doesn't match anything, return the default value (``None`` if the argument is not provided). By default, character entity references are replaced by their corresponding character (except for ``&`` and ``<``. Passing ``replace_entities`` as ``False`` switches off these replacements. c3s|]}|jdVqdS)rNNrPrCrRrr rHz(SelectorList.re_first..)r)rrSdefaultrOelrrRrre_firstts zSelectorList.re_firstcCsdd|DS)z Call the ``.get()`` method for each element is this list and return their results flattened, as a list of unicode strings. cSsg|] }|qSrgetrCrrrrGrHz'SelectorList.getall..rr>rrrgetallszSelectorList.getallcCs|D]}|S|S)z Return the result of ``.get()`` for the first element in this list. If the list is empty, return the default value. rY)rrVrErrrrZs zSelectorList.getcCs|D] }|jSiS)ztReturn the attributes dictionary for the first element. If the list is empty, return an empty dict. )attribrrErrrr\s zSelectorList.attribcCs|D] }|qdS)zU Remove matched nodes from the parent for each element in this list. N)remover]rrrr^szSelectorList.remove)N)T)NT)N)r rr__doc__r4r8r?rBrJrQrXr[extractrZZ extract_firstpropertyr\r^rrrrrr2:s      r2c@seZdZdZgdZdZdddZdZeZ d)dd Z d d Z d*d d Z d+ddZ ddZddZd,ddZd-ddZddZeZddZddZdd Zd!d"Zed#d$Zd%d&ZeZd'd(ZeZdS).Selectoral :class:`Selector` allows you to select parts of an XML or HTML text using CSS or XPath expressions and extract data from it. ``text`` is a ``unicode`` object in Python 2 or a ``str`` object in Python 3 ``type`` defines the selector type, it can be ``"html"``, ``"xml"`` or ``None`` (default). If ``type`` is ``None``, the selector defaults to ``"html"``. ``base_url`` allows setting a URL for the document. This is needed when looking up external entities with relative paths. See [`lxml` documentation](https://lxml.de/api/index.html) ``lxml.etree.fromstring`` for more information. ) r.rAtype_exprr0 __weakref__rrrNz$http://exslt.org/regular-expressionszhttp://exslt.org/sets)rQsetFc Cst|p |j|_}t|d|_t|d|_t|d|_|durxt|tj sjdtj |j f}t || ||}n|durt dt|j|_|dur|j|||_||_dS)Nrrrz*text argument should be of type %s, got %sz+Selector needs either text or root argument)r$ _default_typercr!rrrr9six text_typerr= _get_rootr"dict_default_namespacesrAupdater0rd) rr.rcrAr0r*rdr#msgrrrrs$   zSelector.__init__cCs tddS)Nzcan't pickle Selector objectsr<r>rrrr?szSelector.__getstate__cCst||j|dS)N)r*)r1r)rr.r*rrrrjszSelector._get_rootc sz jj}Wnty(gYS0tj}|durF||z|f|jd|}Wn`tj y}zFd|f}t j r|n| d}t tt|tdWYd}~n d}~00t|tur|g}fdd|D}|S)a Find nodes matching the xpath ``query`` and return the result as a :class:`SelectorList` instance with all elements flattened. List elements implement :class:`Selector` interface too. ``query`` is a string containing the XPATH query to apply. ``namespaces`` is an optional ``prefix: namespace-uri`` mapping (dict) for additional prefixes to those registered with ``register_namespace(prefix, uri)``. Contrary to ``register_namespace()``, these prefixes are not saved for future calls. Any additional named arguments can be used to pass values for XPath variables in the XPath expression, e.g.:: selector.xpath('//a[href=$url]', url="http://www.example.com") N)rAZ smart_stringszXPath error: %s in %sZunicode_escapecs"g|]}j|jjdqS))r0rdrArc)rrArcrCrLrrrrG s z"Selector.xpath..)r0rBAttributeErrorselectorlist_clsrkrArm_lxml_smart_stringsrZ XPathErrorrhZPY3r-Zreraiser"sysexc_inforclist) rrLrArZxpathevZnspresultexcrnrrprrBs.      0  zSelector.xpathcCs|||S)aj Apply the given CSS selector and return a :class:`SelectorList` instance. ``query`` is a string containing the CSS selector to apply. In the background, CSS queries are translated into XPath queries using `cssselect`_ library and run ``.xpath()`` method. .. _cssselect: https://pypi.python.org/pypi/cssselect/ )rB _css2xpathrMrrrrJs z Selector.csscCs |j|Sr3)rZ css_to_xpathrMrrrryszSelector._css2xpathTcCst|||dS)a Apply the given regex and return a list of unicode strings with the matches. ``regex`` can be either a compiled regular expression or a string which will be compiled to a regular expression using ``re.compile(regex)``. By default, character entity references are replaced by their corresponding character (except for ``&`` and ``<``). Passing ``replace_entities`` as ``False`` switches off these replacements. rN)rrZrTrrrrQs z Selector.recCstt|j||d|S)a Apply the given regex and return the first unicode string which matches. If there is no match, return the default value (``None`` if the argument is not provided). By default, character entity references are replaced by their corresponding character (except for ``&`` and ``<``). Passing ``replace_entities`` as ``False`` switches off these replacements. rN)nextrrQ)rrSrVrOrrrrX.s zSelector.re_firstc Csdztj|j|jdddWSttfy^|jdur:YdS|jdurJYdSt|jYSYn0dS)z Serialize and return the matched nodes in a single unicode string. Percent encoded content is unquoted. ZunicodeF)methodr(Z with_tailT10N)rZtostringr0rrqr=rhrir>rrrrZ;s  z Selector.getcCs |gS)z_ Serialize and return the matched node in a 1-element list of unicode strings. rYr>rrrr[NszSelector.getallcCs||j|<dS)z Register the given namespace to be used in this :class:`Selector`. Without registering namespaces you can't select or extract data from non-standard namespaces. See :ref:`selector-examples-xml`. Nr@)rprefixZurirrrregister_namespaceTszSelector.register_namespacecCsz|jdD]\}|jdr0|jddd|_|jD],}|dr:|j||j|ddd<q:q t |jdS)z Remove all namespaces, allowing to traverse the document using namespace-less xpaths. See :ref:`removing-namespaces`. *{}rN) r0itertag startswithsplitr\keyspoprZcleanup_namespaces)rrWZanrrrremove_namespaces\s  "zSelector.remove_namespacescCsZz|j}Wnty(tdYn0z||jWntyTtdYn0dS)z? Remove matched nodes from the parent element. zThe node you're trying to remove has no root, are you trying to remove a pseudo-element? Try to use 'li' as a selector instead of 'li::text' or '//li' instead of '//li/text()', for example.zXThe node you're trying to remove has no parent, are you trying to remove a root element?N)r0Z getparentrqr r^r)rparentrrrr^ks   zSelector.removecCs t|jjS)zAReturn the attributes dictionary for underlying element. )rkr0r\r>rrrr\szSelector.attribcCs t|S)z Return ``True`` if there is any real content selected or ``False`` otherwise. In other words, the boolean value of a :class:`Selector` is given by the contents it selects. )boolrZr>rrr__bool__szSelector.__bool__cCs*tt|dd}dt|j|j|fS)N()widthz<%s xpath=%r data=%s>)reprrrZrcr rd)rdatarrr__str__szSelector.__str__)NNNNNN)N)N)T)NT)r rrr_ __slots__rgrlrsr2rrrr?rjrBrJryrQrXrZr`r[rrr^rar\rZ __nonzero__r__repr__rrrrrbs<     ,    rb)N)r_rtrhZlxmlrrZutilsrrrrZ csstranslatorr r Exceptionr rZ XMLParserrZ HTMLParserr!r$r1rvr2objectrbrrrrs*  m